-- Logs begin at Thu 2022-06-30 09:33:58 UTC, end at Thu 2022-06-30 10:52:19 UTC. -- Jun 30 10:12:43.215373 dv-cinder-820994-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 30 10:12:43.222630 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 30 10:12:43.224677 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 30 10:12:43 2022] *** Jun 30 10:12:43.224677 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 30 10:12:43.224677 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 30 10:12:43.224677 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: nodename: dv-cinder-820994-3-iscsi Jun 30 10:12:43.224677 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: machine: x86_64 Jun 30 10:12:43.224677 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: clock source: unix Jun 30 10:12:43.224677 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: pcre jit disabled Jun 30 10:12:43.224677 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: detected number of CPU cores: 4 Jun 30 10:12:43.224677 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: current working directory: / Jun 30 10:12:43.224677 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: detected binary path: /usr/bin/uwsgi-core Jun 30 10:12:43.225870 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: your processes number limit is 47680 Jun 30 10:12:43.225870 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: your memory page size is 4096 bytes Jun 30 10:12:43.225870 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: detected max file descriptor number: 2048 Jun 30 10:12:43.225870 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: lock engine: pthread robust mutexes Jun 30 10:12:43.225870 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: thunder lock: enabled Jun 30 10:12:43.225870 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 30 10:12:43.225870 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 30 10:12:43.273806 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: Python main interpreter initialized at 0x557d299509c0 Jun 30 10:12:43.273806 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: python threads support enabled Jun 30 10:12:43.273806 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: your server socket listen backlog is limited to 100 connections Jun 30 10:12:43.273806 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: your mercy for graceful operations on workers is 90 seconds Jun 30 10:12:43.274194 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: mapped 403077 bytes (393 KB) for 2 cores Jun 30 10:12:43.274326 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: *** Operational MODE: preforking *** Jun 30 10:12:43.274606 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: *** uWSGI is running in multiple interpreter mode *** Jun 30 10:12:43.274606 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: spawned uWSGI master process (pid: 80349) Jun 30 10:12:43.274812 dv-cinder-820994-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 30 10:12:43.276049 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: spawned uWSGI worker 1 (pid: 80363, cores: 1) Jun 30 10:12:43.276049 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: spawned uWSGI worker 2 (pid: 80364, cores: 1) Jun 30 10:12:43.276049 dv-cinder-820994-3-iscsi devstack@keystone.service[80349]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 30 10:12:48.796387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 10:12:48.796387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: from cryptography.utils import int_from_bytes Jun 30 10:12:48.797125 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 10:12:48.797125 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: from cryptography.utils import int_from_bytes Jun 30 10:12:48.797125 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 10:12:48.797125 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: from cryptography.utils import int_from_bytes Jun 30 10:12:48.797648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 30 10:12:48.797648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: from cryptography.utils import int_from_bytes Jun 30 10:12:48.845333 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 30 10:12:48.845512 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 30 10:12:48.845702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] command line args: [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 30 10:12:48.845825 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 30 10:12:48.845971 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 30 10:12:48.846056 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 30 10:12:48.846156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ================================================================================ {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 30 10:12:48.846241 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] command line args: [] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 30 10:12:48.846410 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] admin_token = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.846495 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 30 10:12:48.846634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ================================================================================ {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 30 10:12:48.846717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] config_dir = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.846839 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.846914 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] admin_token = **** {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.847024 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] config_source = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.847123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] config_dir = [] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.847203 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] debug = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.847334 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.847420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.847714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] config_source = [] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.847714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] debug = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.847909 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.848008 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.848126 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.848230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.848341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.848441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.848534 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.848632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.848726 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.848835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.848936 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_config_append = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.849035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] list_limit = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.849128 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.849226 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] log_config_append = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.849323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_dir = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.849428 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.849533 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_file = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.849625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] log_dir = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.849807 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.850427 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] log_file = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.850509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.850635 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.850721 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.850851 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.850942 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.851013 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.851138 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.851220 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.851317 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.851393 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.851509 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.851583 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.851708 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.851785 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.851951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.852037 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.852187 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.852272 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.852423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.852506 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.852657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.852732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.852894 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.852978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.853140 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.853224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.853387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.853502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.853662 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.853892 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.47/identity {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.854080 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] publish_errors = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.854198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.47/identity {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.854314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] publish_errors = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.854398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.854547 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.854622 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.854728 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.854803 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.854949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.855030 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.855197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.855295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.855460 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] standard_threads = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.855536 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.855695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.855771 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] standard_threads = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.855928 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.856003 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.856139 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.856224 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.856301 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] use_journal = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.856371 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.856439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] use_json = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.856513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] use_journal = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.856667 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] use_stderr = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.856741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] use_json = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.856810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] use_syslog = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.856879 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] use_stderr = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.856977 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.857060 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] use_syslog = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.857212 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 30 10:12:48.857293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.857460 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.857541 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.857699 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.857775 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.857951 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.858032 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.858191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.858266 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.858432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.858512 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.858669 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.858750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.858914 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.858995 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.859142 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.859248 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.859413 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.859498 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.859661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.859737 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.859894 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.859968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.860133 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.860209 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.860370 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.860449 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.860606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.860682 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.860855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.860932 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.861093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.861168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.861328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.861420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.861583 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.861669 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.861823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.861898 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.862060 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.862211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.862285 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.862361 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.862533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.862663 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.862900 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.862984 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.863156 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.863232 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.863429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.863505 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.863661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.863748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.863894 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.863970 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.864126 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.864208 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.864363 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.864454 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.864612 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.864688 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.864844 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.864919 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.865081 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.865167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.865339 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.865427 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.865598 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.865706 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.865865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.865980 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.866198 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.866301 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.866437 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.866573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.866678 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.866783 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.866894 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.867006 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.867129 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.867208 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.867390 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.867479 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.867594 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.867668 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.867829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.867915 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.868061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.868189 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.868300 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.868389 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.868532 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.868609 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.868771 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.868849 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.869003 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.869082 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.869241 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.869323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.869476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.869558 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.869714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.869791 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.869947 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.870021 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.870184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.870266 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.870420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.870496 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.870659 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.870733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.870895 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.870970 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.871132 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.871208 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.871393 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.871473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.871627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.871702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.871871 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.871948 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.872107 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.872188 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.872340 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.872424 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.872572 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.872654 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.872828 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.872905 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.873062 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.873138 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.873294 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.873376 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.873529 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.873605 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.873768 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.873848 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.874026 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.874100 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.874262 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.874344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.874510 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.874591 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.874750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.874828 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.874983 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.875057 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.875217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.875340 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.875477 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.875563 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.875716 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.875798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.875952 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.876027 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.876190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.876272 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.876426 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.876504 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.876661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.876737 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.876896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.876982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.877128 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.877204 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.877363 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.877438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.877598 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.877686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.877850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.877932 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.878086 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.878166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.878321 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.878404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.878556 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.878631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.878792 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.878873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.879026 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.879109 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.879277 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.879365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.879518 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.879630 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.879750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.879869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.879981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.880101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.880217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.880331 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.880447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.880565 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.880687 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.880796 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.880941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.881079 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.881216 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.881360 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.881492 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.881573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.881644 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.881772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.881904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.882025 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.882145 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.882283 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.882360 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.external = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.882525 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.882602 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.882765 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] auth.external = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.882877 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.883003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.883106 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.883319 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.883403 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.password = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.883556 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.883634 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] auth.token = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.883788 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] auth.password = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.883867 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.884019 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] auth.token = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.884096 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.884258 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.884335 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.884489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.884572 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.884733 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.884809 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.884965 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.885051 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.885212 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.885289 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.885452 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.885529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.885681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.885762 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.885917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] credential.caching = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.886002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.886165 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.886241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.886402 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.886478 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.886655 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.886732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.886889 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.886971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.887132 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.887208 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.887390 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.887470 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.887648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.887724 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.887900 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.887991 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.888136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.888211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.888379 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.888457 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.888661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.888738 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.888902 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.888979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.889160 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.889238 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.889397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.889473 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.889636 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.889715 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.889872 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] federation.caching = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.889948 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.890111 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.890187 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.890340 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.890417 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.890587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.890665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.890815 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.890899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.891062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.891141 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.891317 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.891398 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.891576 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.891665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.891815 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.891894 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.892064 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.892141 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.892301 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.892388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.892548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.892626 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.892788 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.caching = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.892863 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.893021 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.893106 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.893257 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.893334 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.893492 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.893568 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.893721 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.893808 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.893960 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.894037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.894188 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.894264 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.894427 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.894512 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.894666 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.894748 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.894898 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.894975 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.895129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.895214 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.895380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.895468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.895614 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.895692 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.896110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.896201 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.896370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.896448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.897136 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.897251 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.897484 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.897640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.897723 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.897986 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.898079 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.898225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.898303 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.898466 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.898544 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.898698 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.898774 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.898936 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.899012 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.899169 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.899247 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.899428 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.899507 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.899671 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.899749 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.899909 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.899985 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.900142 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.900224 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.900373 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.900457 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.900621 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.900698 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.900857 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.900932 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.901088 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.901172 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.901333 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.901412 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.901562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.901638 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.901796 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.901880 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.902026 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.902103 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.902255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.902330 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.902491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.902574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.902726 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.902803 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.902962 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.903037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.903198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.903300 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.903459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.903537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.903708 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.903785 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.903942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.904029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.904175 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.904253 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.904404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.904479 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.904651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.904735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.904886 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.904963 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.905121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.905198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.905355 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.905438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.905584 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.905660 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.905812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.905888 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.906053 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.906138 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.906289 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.906365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.906538 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.906624 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.906783 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.906867 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.907027 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.907103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.907303 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.907384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.907514 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.907604 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.907742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.907818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.907975 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.908052 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.908207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.908292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.908437 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.908513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.908664 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.908739 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.908908 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.908993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.909152 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.909228 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.909398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.909474 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.909642 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.909726 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.909869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.909945 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.910106 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.910181 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.910341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.910425 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.910578 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.910655 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.910813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.910889 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.911047 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.911130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.911298 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.911378 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.911550 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.911627 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.911786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.911949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.912025 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.912095 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.912838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.912974 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.913088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.913175 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.913323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.913516 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.913593 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.913759 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.913836 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.913997 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.914075 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.914237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.914320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.914474 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.914550 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.914713 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.914791 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.914943 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.915018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.915179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.915256 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.915453 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.915531 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] resource.caching = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.915692 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.915769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.915937 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.916013 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.916173 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.916252 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.916419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.916496 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.916667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.916750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.916903 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.916979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.917142 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.917218 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.917375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] role.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.917460 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.917612 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.917688 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] role.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.917841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.917916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] role.caching = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.918083 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] role.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.918166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.918313 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.918390 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.918560 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.918638 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.918804 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.918888 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.919048 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.919125 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.919313 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.919393 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.919562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.919650 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.919807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.919884 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.920037 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.920113 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.920283 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.920367 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.920512 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.920593 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.920757 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.920834 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.921001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.921085 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.921245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.921395 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.921492 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.921635 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.921882 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.921967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.922118 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.922196 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.922365 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.922441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.922614 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.922692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.922850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.922927 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.923103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.923186 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.923360 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.923440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.923612 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.923689 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.923846 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.923924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.924079 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.924158 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.924309 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.924385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.924541 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.924619 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.924771 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.924850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.925020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.925096 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.925257 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.925333 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.925494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.925572 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.925725 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.925801 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.925957 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.926033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.926188 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.926266 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.926422 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] token.caching = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.926505 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.926658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.926736 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.926902 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.926981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.927135 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.927211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.927397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.927476 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.927642 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.927721 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.927914 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.928028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.928160 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.928397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.928483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.928638 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.928716 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.928873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.928950 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.929119 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.929205 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.929368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.929444 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.929597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.929679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.929839 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.929925 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.930087 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.930165 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.930317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.930394 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.connection = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.930560 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.930647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.930852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.930931 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.931090 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.931168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.connection = **** {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.931359 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.931454 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.931598 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.931676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.931820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.931893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.932059 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.932137 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.932293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.932370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.932527 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.932605 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.932763 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.932839 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.932996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.933072 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.933230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.933308 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.933468 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.933545 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.933702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.933779 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.933935 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.934012 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.934186 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.934273 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.934401 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.934478 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.934636 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.934714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.934806 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80363) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 30 10:12:48.934895 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.935089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.935346 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 30 10:12:48.935519 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80364) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 30 10:12:48.965843 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:48.966075 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:48.966303 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:48.966605 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:48.966843 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:48.967079 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:48.974768 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:48.975470 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.002659 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.002891 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.003232 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.003494 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.145070 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.145286 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.195556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.195790 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.196123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.196363 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.259760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.259989 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.260117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.260206 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.260351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.260439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.260576 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.260668 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.260796 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.260878 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.261015 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.261231 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.285591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.285823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.285907 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.286048 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.286132 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.286268 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.286353 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.286482 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.286565 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.286694 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.286777 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.286907 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.286991 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.287732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.298532 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.298751 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.311281 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.311520 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.311651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.311781 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.311908 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.312067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.312194 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.312325 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.312452 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.312578 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.312705 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.312851 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80363) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.312979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.312979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80364) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 30 10:12:49.330050 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.330204 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.330347 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 10:12:49.330476 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 10:12:49.330604 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.330731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.330857 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 10:12:49.330991 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 30 10:12:49.331132 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.331259 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.331406 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.331537 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.331667 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.331807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.331938 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.332066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.343177 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.343339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.343481 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.343609 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.343746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.343874 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.344000 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.344128 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.352822 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.352967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.356102 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.356282 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.356409 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.356552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.356694 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.356827 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.356958 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.357089 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.357227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.357358 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.357487 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.357616 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.357744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.357875 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.358013 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.358154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.358283 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.358413 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.358542 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.358679 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.358810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.358941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.384890 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.385263 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.386493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.386771 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.386898 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.387131 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.392028 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.392307 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.392446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.392639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.392769 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.393033 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.400866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.401099 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.401240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.401524 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.405553 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.405842 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.405979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.406112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.406270 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.406398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.406552 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.406680 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.406825 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.406963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.407095 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.407225 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.407373 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.407510 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.407638 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.407873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.426500 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.426777 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.426949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.427158 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.431520 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.431836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.435108 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.435424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.435522 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.435728 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.440509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.440814 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.444118 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.444540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.447763 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.448060 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.448178 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.448349 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.448476 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.448643 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.448761 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.448938 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.449051 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.449220 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.449341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.449625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.469070 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.469354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.469618 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.469878 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.469960 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.470193 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.470462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.470724 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.478643 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.480387 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.480664 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.482959 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.483893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.484175 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.484456 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.484717 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.484814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.485472 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.485575 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.488362 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.488623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.488903 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.489261 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.489771 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.498436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.500985 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.500985 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.500985 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.500985 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.501595 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.501595 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.503839 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.504131 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.504418 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.504699 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.504979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.506741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.507043 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.519371 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.523402 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.525592 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.526695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.528449 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.528865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.530076 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.530355 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.530642 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.532183 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.533892 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.535543 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.535820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.536106 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.538497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.540132 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.543358 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.543658 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.544048 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.544187 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.544316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.545741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.548979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.549279 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.549670 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.549948 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.553896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.554185 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.554466 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.559847 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.560136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.560420 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.562249 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80363) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.562528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.562806 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.563079 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.563373 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.563762 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.564046 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.564327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.564619 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.564895 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.565174 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.565438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.565703 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.565979 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80363) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.568217 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80364) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 30 10:12:49.569213 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.569544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.569858 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.570130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.570520 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.570812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.571085 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.571392 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.571666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.571944 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.572208 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.572472 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.572736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80364) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 30 10:12:49.604317 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557d299509c0 pid: 80363 (default app) Jun 30 10:12:49.607732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffc677-16cc-49d4-95bd-768f9b8b1dfa None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:12:49.608027 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffc677-16cc-49d4-95bd-768f9b8b1dfa None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:12:49.608306 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffc677-16cc-49d4-95bd-768f9b8b1dfa None None] PATH_INFO: `/v3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:12:49.610350 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x557d299509c0 pid: 80364 (default app) Jun 30 10:12:49.612559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1/1] 10.222.0.47 () {50 vars in 791 bytes} [Thu Jun 30 10:12:49 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:12:58.361822 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b2fca-2122-4a1a-bc10-32750c65df97 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:12:58.362392 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b2fca-2122-4a1a-bc10-32750c65df97 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:12:58.362486 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60b2fca-2122-4a1a-bc10-32750c65df97 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:12:58.367375 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1/2] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:12:58 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:12:58.372594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1a1ce-7206-433c-8c64-3eff5a5d8728 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:12:58.372983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1a1ce-7206-433c-8c64-3eff5a5d8728 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:12:58.373149 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e1a1ce-7206-433c-8c64-3eff5a5d8728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:12:58.452404 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: INFO dbcounter [None req-b2e1a1ce-7206-433c-8c64-3eff5a5d8728 None None] Registered counter for database keystone Jun 30 10:12:58.475649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG oslo_db.sqlalchemy.engines [None req-b2e1a1ce-7206-433c-8c64-3eff5a5d8728 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80363) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 30 10:12:58.482025 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writer thread running {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 30 10:12:58.673226 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG passlib.handlers.bcrypt [None req-b2e1a1ce-7206-433c-8c64-3eff5a5d8728 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80363) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 30 10:12:58.673644 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG passlib.handlers.bcrypt [None req-b2e1a1ce-7206-433c-8c64-3eff5a5d8728 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80363) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 30 10:12:58.731466 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b2e1a1ce-7206-433c-8c64-3eff5a5d8728 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:12:58.733077 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b2e1a1ce-7206-433c-8c64-3eff5a5d8728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:12:58.856933 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 2/3] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:12:58 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 485 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 30 10:12:59.046379 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b2fca-2122-4a1a-bc10-32750c65df97 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:12:59.116983 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: INFO dbcounter [None req-b60b2fca-2122-4a1a-bc10-32750c65df97 None None] Registered counter for database keystone Jun 30 10:12:59.136186 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG oslo_db.sqlalchemy.engines [None req-b60b2fca-2122-4a1a-bc10-32750c65df97 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80364) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 30 10:12:59.142432 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writer thread running {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 30 10:12:59.263176 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60b2fca-2122-4a1a-bc10-32750c65df97 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:12:59.267487 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b33f89-4e55-494c-9da0-1dbb0784eaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:12:59.268151 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b33f89-4e55-494c-9da0-1dbb0784eaaa admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:12:59.268441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b33f89-4e55-494c-9da0-1dbb0784eaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:12:59.268721 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b33f89-4e55-494c-9da0-1dbb0784eaaa admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:12:59.277933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b33f89-4e55-494c-9da0-1dbb0784eaaa admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:12:59.329814 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: WARNING keystone.common.rbac_enforcer.enforcer [None req-65b33f89-4e55-494c-9da0-1dbb0784eaaa admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 30 10:12:59.330096 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b33f89-4e55-494c-9da0-1dbb0784eaaa admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:12:59.339797 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 2/4] 10.222.0.47 () {58 vars in 1324 bytes} [Thu Jun 30 10:12:59 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:12:59.345799 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e1a1ce-7206-433c-8c64-3eff5a5d8728 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:12:59.360130 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e1a1ce-7206-433c-8c64-3eff5a5d8728 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:12:59.364538 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc49f0-abbb-4786-86dd-eb664579fa2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:12:59.366198 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc49f0-abbb-4786-86dd-eb664579fa2c admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:12:59.366483 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc49f0-abbb-4786-86dd-eb664579fa2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:12:59.366761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc49f0-abbb-4786-86dd-eb664579fa2c admin admin] PATH_INFO: `/v3/projects/429c0e046ffb41cbb44280918656fef0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:12:59.368314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc49f0-abbb-4786-86dd-eb664579fa2c admin admin] RBAC: Authorizing `identity:get_project(project_id=429c0e046ffb41cbb44280918656fef0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:12:59.421763 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: WARNING keystone.common.rbac_enforcer.enforcer [None req-f2cc49f0-abbb-4786-86dd-eb664579fa2c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 30 10:12:59.422323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc49f0-abbb-4786-86dd-eb664579fa2c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:12:59.426672 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 3/5] 10.222.0.47 () {56 vars in 1207 bytes} [Thu Jun 30 10:12:59 2022] GET /identity/v3/projects/429c0e046ffb41cbb44280918656fef0 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:01.444335 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b897d-7901-4ee4-9213-b15905ab6864 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:01.444921 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b897d-7901-4ee4-9213-b15905ab6864 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:01.444921 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4b897d-7901-4ee4-9213-b15905ab6864 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:01.449041 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 3/6] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:13:01.454483 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3d765-51bf-493c-a61c-acfe013e3268 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:01.454911 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3d765-51bf-493c-a61c-acfe013e3268 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:01.455095 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3d765-51bf-493c-a61c-acfe013e3268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:01.473592 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7ed3d765-51bf-493c-a61c-acfe013e3268 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:01.475133 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7ed3d765-51bf-493c-a61c-acfe013e3268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:01.483064 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 4/7] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:01 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 30 10:13:01.506800 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b897d-7901-4ee4-9213-b15905ab6864 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:01.518864 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4b897d-7901-4ee4-9213-b15905ab6864 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:01.523441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6c901-8cf3-49f7-b767-9a9def6f7854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:01.524082 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6c901-8cf3-49f7-b767-9a9def6f7854 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:01.524371 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6c901-8cf3-49f7-b767-9a9def6f7854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:01.524654 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6c901-8cf3-49f7-b767-9a9def6f7854 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:01.532366 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6c901-8cf3-49f7-b767-9a9def6f7854 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:01.534154 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6c901-8cf3-49f7-b767-9a9def6f7854 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:01.542942 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 4/8] 10.222.0.47 () {58 vars in 1324 bytes} [Thu Jun 30 10:13:01 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:13:01.548302 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3d765-51bf-493c-a61c-acfe013e3268 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:01.558110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3d765-51bf-493c-a61c-acfe013e3268 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:01.562539 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1041381-70e2-48a1-85d0-89679df774e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:01.563448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1041381-70e2-48a1-85d0-89679df774e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:01.563739 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1041381-70e2-48a1-85d0-89679df774e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:01.564019 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1041381-70e2-48a1-85d0-89679df774e8 admin admin] PATH_INFO: `/v3/users/67e761d82cde42509f040f82a44c86c4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:01.565435 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1041381-70e2-48a1-85d0-89679df774e8 admin admin] RBAC: Authorizing `identity:get_user(user_id=67e761d82cde42509f040f82a44c86c4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:01.567327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1041381-70e2-48a1-85d0-89679df774e8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:01.569771 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 5/9] 10.222.0.47 () {56 vars in 1198 bytes} [Thu Jun 30 10:13:01 2022] GET /identity/v3/users/67e761d82cde42509f040f82a44c86c4 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:03.657740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410e923-f63c-4ddd-be9b-87cd8f008870 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:03.658280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410e923-f63c-4ddd-be9b-87cd8f008870 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:03.658280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5410e923-f63c-4ddd-be9b-87cd8f008870 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:03.660021 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 5/10] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:03.666333 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b65b50-04a8-47d3-ba69-2ab8cf7ab36a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:03.666764 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b65b50-04a8-47d3-ba69-2ab8cf7ab36a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:03.667065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b65b50-04a8-47d3-ba69-2ab8cf7ab36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:03.683740 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-01b65b50-04a8-47d3-ba69-2ab8cf7ab36a None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:03.684929 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-01b65b50-04a8-47d3-ba69-2ab8cf7ab36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:03.712832 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce356f27-d05a-4366-8266-3a286411f6e0 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:03.713122 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce356f27-d05a-4366-8266-3a286411f6e0 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:03.713402 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce356f27-d05a-4366-8266-3a286411f6e0 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:03.714754 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 6/11] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:13:03.720150 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ef847-fdb0-470b-8704-5d2dfc19bebb None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:03.720440 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ef847-fdb0-470b-8704-5d2dfc19bebb None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:03.720721 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8ef847-fdb0-470b-8704-5d2dfc19bebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:03.768265 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG passlib.handlers.bcrypt [None req-4d8ef847-fdb0-470b-8704-5d2dfc19bebb None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80364) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 30 10:13:03.768877 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG passlib.handlers.bcrypt [None req-4d8ef847-fdb0-470b-8704-5d2dfc19bebb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80364) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 30 10:13:03.782915 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 6/12] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 10:13:03.806398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b65b50-04a8-47d3-ba69-2ab8cf7ab36a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:03.817446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-4d8ef847-fdb0-470b-8704-5d2dfc19bebb None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:03.819653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-4d8ef847-fdb0-470b-8704-5d2dfc19bebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:03.844853 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:03.929313 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 7/13] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 10:13:03.935359 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b65b50-04a8-47d3-ba69-2ab8cf7ab36a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:03.952409 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ef847-fdb0-470b-8704-5d2dfc19bebb None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:03.971558 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d66e6b-e245-4e21-b831-884cb7494ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:03.972430 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d66e6b-e245-4e21-b831-884cb7494ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:03.972726 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d66e6b-e245-4e21-b831-884cb7494ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:03.973026 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d66e6b-e245-4e21-b831-884cb7494ab6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:03.985662 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d66e6b-e245-4e21-b831-884cb7494ab6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:03.997900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-d5d66e6b-e245-4e21-b831-884cb7494ab6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:13:03.997900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:03.997900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:03.997900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:03.997900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:03.997900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:03.997900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:03.997900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:03.997900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:03.997900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:03.997900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:03.997900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:03.997900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:03.997900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:03.997900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:03.997900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:03.998855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:03.998855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:03.998855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:03.998855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:03.998855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:03.998855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:03.998855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:03.998855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:03.998855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:03.998855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:03.998855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:03.998855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:03.998855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:03.998855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:03.998855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:03.998855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:03.999786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:03.999786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:03.999786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:03.999786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:03.999786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:03.999786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:03.999786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:03.999786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:03.999786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:03.999786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:03.999786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:13:03.999786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:13:03.999786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 7/14] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:13:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:04.005031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d66e6b-e245-4e21-b831-884cb7494ab6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:04.069000 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8ef847-fdb0-470b-8704-5d2dfc19bebb None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:04.072046 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:04.097146 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:04.108322 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe236fe-5fce-4c00-a091-cf5105699295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:04.109218 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe236fe-5fce-4c00-a091-cf5105699295 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:04.109505 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe236fe-5fce-4c00-a091-cf5105699295 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:04.109779 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe236fe-5fce-4c00-a091-cf5105699295 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:04.116095 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d66e6b-e245-4e21-b831-884cb7494ab6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:04.149953 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378f673-0908-452d-890c-b0bfccc64ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:04.150792 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378f673-0908-452d-890c-b0bfccc64ca9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:04.151165 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378f673-0908-452d-890c-b0bfccc64ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:04.151567 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6378f673-0908-452d-890c-b0bfccc64ca9 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:04.152628 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6378f673-0908-452d-890c-b0bfccc64ca9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:04.153057 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6378f673-0908-452d-890c-b0bfccc64ca9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:04.154992 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6378f673-0908-452d-890c-b0bfccc64ca9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:04.175788 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 8/15] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:04.178207 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378f673-0908-452d-890c-b0bfccc64ca9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:04.179381 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe236fe-5fce-4c00-a091-cf5105699295 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:04.180912 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe236fe-5fce-4c00-a091-cf5105699295 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:04.279060 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 8/16] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:13:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 10:13:04.280127 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6378f673-0908-452d-890c-b0bfccc64ca9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:04.284712 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe236fe-5fce-4c00-a091-cf5105699295 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:04.288468 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:04.315055 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f854579f-e744-437a-8a42-bc9d9a1bb1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:04.315979 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854579f-e744-437a-8a42-bc9d9a1bb1a7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:04.316263 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854579f-e744-437a-8a42-bc9d9a1bb1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:04.316540 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f854579f-e744-437a-8a42-bc9d9a1bb1a7 None admin] PATH_INFO: `/v3/users/67e761d82cde42509f040f82a44c86c4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:04.317787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f854579f-e744-437a-8a42-bc9d9a1bb1a7 None admin] RBAC: Authorizing `identity:get_user(user_id=67e761d82cde42509f040f82a44c86c4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:04.319355 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f854579f-e744-437a-8a42-bc9d9a1bb1a7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:04.321001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 9/17] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:04 2022] GET /identity/v3/users/67e761d82cde42509f040f82a44c86c4 => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:04.326282 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f854579f-e744-437a-8a42-bc9d9a1bb1a7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:04.341886 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:04.388101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe236fe-5fce-4c00-a091-cf5105699295 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:04.425798 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f854579f-e744-437a-8a42-bc9d9a1bb1a7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:04.429187 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edea363-4a5c-4c27-a6bc-c011436055b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:04.429989 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edea363-4a5c-4c27-a6bc-c011436055b6 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:04.430277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edea363-4a5c-4c27-a6bc-c011436055b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:04.430558 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edea363-4a5c-4c27-a6bc-c011436055b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:04.448756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edea363-4a5c-4c27-a6bc-c011436055b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:04.450245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edea363-4a5c-4c27-a6bc-c011436055b6 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:04.461249 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe639f8b-4349-45d3-a151-0dd9fe3a355c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:04.462047 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe639f8b-4349-45d3-a151-0dd9fe3a355c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:04.462328 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe639f8b-4349-45d3-a151-0dd9fe3a355c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:04.462602 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe639f8b-4349-45d3-a151-0dd9fe3a355c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:04.463854 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe639f8b-4349-45d3-a151-0dd9fe3a355c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:04.465451 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe639f8b-4349-45d3-a151-0dd9fe3a355c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:04.467562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 10/18] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:13:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:04.475573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe639f8b-4349-45d3-a151-0dd9fe3a355c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:04.476239 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-0edea363-4a5c-4c27-a6bc-c011436055b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:13:04.476239 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:04.476239 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:04.476239 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:04.476239 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:04.476239 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:04.476239 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:04.476239 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:04.476239 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:04.476239 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:04.476239 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:04.476239 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:04.476239 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:13:04.476239 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:13:04.476239 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:13:04.477384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:13:04.477384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:04.477384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:04.477384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:04.477384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:04.477384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:04.477384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:04.477384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:04.477384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:04.477384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:04.477384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:04.477384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:04.477384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:04.477384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:04.477384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:04.477384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:04.477384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:04.478178 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:13:04.478178 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:13:04.478178 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:13:04.478178 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:13:04.478178 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 9/19] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:13:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:13:04.484316 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edea363-4a5c-4c27-a6bc-c011436055b6 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:04.544512 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:04.561459 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:04.581691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe639f8b-4349-45d3-a151-0dd9fe3a355c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:04.601581 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edea363-4a5c-4c27-a6bc-c011436055b6 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:04.616055 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca7a038-790f-465f-a688-b8457f51e060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:04.616719 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7a038-790f-465f-a688-b8457f51e060 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:04.617011 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7a038-790f-465f-a688-b8457f51e060 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:04.617298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7a038-790f-465f-a688-b8457f51e060 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:04.618143 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca7a038-790f-465f-a688-b8457f51e060 None admin] RBAC: Adding query filter params (role.id=e63bda042e104a4a93fba15ffcfe1183, scope.domain.id=default, user.id=67e761d82cde42509f040f82a44c86c4) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:04.618468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca7a038-790f-465f-a688-b8457f51e060 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:04.619983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca7a038-790f-465f-a688-b8457f51e060 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:04.626682 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 11/20] 10.222.0.47 () {56 vars in 1322 bytes} [Thu Jun 30 10:13:04 2022] GET /identity/v3/role_assignments?user.id=67e761d82cde42509f040f82a44c86c4&scope.domain.id=default&role.id=e63bda042e104a4a93fba15ffcfe1183 => generated 233 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:04.639932 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66830039-335f-4142-958d-1b4ca03b4e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:04.640622 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66830039-335f-4142-958d-1b4ca03b4e8c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:04.640906 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66830039-335f-4142-958d-1b4ca03b4e8c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:04.641181 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66830039-335f-4142-958d-1b4ca03b4e8c None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:04.642190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66830039-335f-4142-958d-1b4ca03b4e8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:04.642499 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66830039-335f-4142-958d-1b4ca03b4e8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:04.643984 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66830039-335f-4142-958d-1b4ca03b4e8c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:04.667769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 10/21] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:07.001565 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e65ff1-f4a5-4caa-a378-b5f4dc08c3b0 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:07.002632 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e65ff1-f4a5-4caa-a378-b5f4dc08c3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:07.003113 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e65ff1-f4a5-4caa-a378-b5f4dc08c3b0 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:07.004708 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 12/22] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:07.011124 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423ad58-806f-4f96-9fc2-a913970b3d52 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:07.012049 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423ad58-806f-4f96-9fc2-a913970b3d52 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:07.012530 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423ad58-806f-4f96-9fc2-a913970b3d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:07.035808 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-7423ad58-806f-4f96-9fc2-a913970b3d52 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:07.037640 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-7423ad58-806f-4f96-9fc2-a913970b3d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:07.133883 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 11/23] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:07.181080 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8236bf-684d-430b-90db-484db7bc9fc7 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:07.181760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8236bf-684d-430b-90db-484db7bc9fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:07.182275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8236bf-684d-430b-90db-484db7bc9fc7 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:07.184181 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 13/24] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:07.185616 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423ad58-806f-4f96-9fc2-a913970b3d52 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:07.189586 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcfda5-92d9-4fba-a742-f6232bdac91c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:07.190199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcfda5-92d9-4fba-a742-f6232bdac91c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:07.190723 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbcfda5-92d9-4fba-a742-f6232bdac91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:07.222276 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8cbcfda5-92d9-4fba-a742-f6232bdac91c None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:07.225170 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8cbcfda5-92d9-4fba-a742-f6232bdac91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:07.263399 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:07.319509 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423ad58-806f-4f96-9fc2-a913970b3d52 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:07.377153 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc463b-f623-4860-b8d9-28c7e3390a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:07.378020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc463b-f623-4860-b8d9-28c7e3390a11 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:07.378306 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc463b-f623-4860-b8d9-28c7e3390a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:07.378588 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfc463b-f623-4860-b8d9-28c7e3390a11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:07.396936 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfc463b-f623-4860-b8d9-28c7e3390a11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:07.398661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfc463b-f623-4860-b8d9-28c7e3390a11 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:07.399108 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 14/25] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 10:13:07.427807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-abfc463b-f623-4860-b8d9-28c7e3390a11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:13:07.427807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:07.427807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:07.427807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:07.427807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:07.427807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:07.427807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:07.427807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:07.427807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:07.427807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:07.427807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:07.427807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:07.427807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:13:07.427807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:13:07.427807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:13:07.428697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:13:07.428697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:07.428697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:07.428697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:07.428697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:07.428697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:07.428697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:07.428697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:07.428697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:07.428697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:07.428697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:07.428697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:07.428697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:07.428697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:07.428697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:07.428697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:07.428697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:07.429484 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:13:07.429484 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:13:07.429484 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:13:07.429484 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:13:07.429484 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 12/26] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:13:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:07.434780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbcfda5-92d9-4fba-a742-f6232bdac91c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:07.435717 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc463b-f623-4860-b8d9-28c7e3390a11 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:07.576488 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:07.581713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfc463b-f623-4860-b8d9-28c7e3390a11 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:07.629781 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5a49a-0e75-4b3b-8e55-c771262b2cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:07.632423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5a49a-0e75-4b3b-8e55-c771262b2cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:07.632762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5a49a-0e75-4b3b-8e55-c771262b2cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:07.633096 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb5a49a-0e75-4b3b-8e55-c771262b2cc0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:07.639037 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:07.655290 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb5a49a-0e75-4b3b-8e55-c771262b2cc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:07.663787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-5cb5a49a-0e75-4b3b-8e55-c771262b2cc0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:13:07.663787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:07.663787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:07.663787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:07.663787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:07.663787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:07.663787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:07.663787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:07.663787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:07.663787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:07.663787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:07.663787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:07.663787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:07.663787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:07.663787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:07.663787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:07.664667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:07.664667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:07.664667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:07.664667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:07.664667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:07.664667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:07.664667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:07.664667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:07.664667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:07.664667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:07.664667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:07.664667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:07.664667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:07.664667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:07.664667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:07.664667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:07.665667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:07.665667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:07.665667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:07.665667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:07.665667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:07.665667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:07.665667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:07.665667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:07.665667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:07.665667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:07.665667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:13:07.665667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:13:07.665667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 13/27] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:13:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:13:07.671170 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5a49a-0e75-4b3b-8e55-c771262b2cc0 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:07.680066 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbcfda5-92d9-4fba-a742-f6232bdac91c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:07.728578 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829a547-dcf3-4e4f-82f2-fcdcf08cfb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:07.731965 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829a547-dcf3-4e4f-82f2-fcdcf08cfb3e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:07.732467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829a547-dcf3-4e4f-82f2-fcdcf08cfb3e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:07.732944 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829a547-dcf3-4e4f-82f2-fcdcf08cfb3e None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:07.733907 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a829a547-dcf3-4e4f-82f2-fcdcf08cfb3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:07.736059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a829a547-dcf3-4e4f-82f2-fcdcf08cfb3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:07.737736 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a829a547-dcf3-4e4f-82f2-fcdcf08cfb3e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:07.764765 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 15/28] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:13:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:07.766675 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4638e-0571-4621-85ea-8a96979ed0ee None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:07.767166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4638e-0571-4621-85ea-8a96979ed0ee None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:07.767692 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc4638e-0571-4621-85ea-8a96979ed0ee None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:07.769836 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 16/29] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:07.771532 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4638e-0571-4621-85ea-8a96979ed0ee None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:07.845468 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb5a49a-0e75-4b3b-8e55-c771262b2cc0 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:07.909128 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74e3bd0-51e6-4e62-9653-d258e73c7785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:07.909985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e3bd0-51e6-4e62-9653-d258e73c7785 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:07.910272 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e3bd0-51e6-4e62-9653-d258e73c7785 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:07.910552 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74e3bd0-51e6-4e62-9653-d258e73c7785 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:07.911329 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e3bd0-51e6-4e62-9653-d258e73c7785 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:07.912184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e3bd0-51e6-4e62-9653-d258e73c7785 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:07.913987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74e3bd0-51e6-4e62-9653-d258e73c7785 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:07.923210 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc4638e-0571-4621-85ea-8a96979ed0ee None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:07.932052 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 14/30] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:07.933564 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9437f9-83d6-4a1a-97b0-bb1dd7504f18 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:07.934036 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9437f9-83d6-4a1a-97b0-bb1dd7504f18 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:07.934367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9437f9-83d6-4a1a-97b0-bb1dd7504f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:07.954728 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-7c9437f9-83d6-4a1a-97b0-bb1dd7504f18 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:07.956137 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-7c9437f9-83d6-4a1a-97b0-bb1dd7504f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:07.967554 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc0e47-b0d3-49ac-99c9-8ee266bc7221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:07.968572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc0e47-b0d3-49ac-99c9-8ee266bc7221 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:07.969056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc0e47-b0d3-49ac-99c9-8ee266bc7221 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:07.969505 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc0e47-b0d3-49ac-99c9-8ee266bc7221 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:07.970588 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc0e47-b0d3-49ac-99c9-8ee266bc7221 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:07.972367 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc0e47-b0d3-49ac-99c9-8ee266bc7221 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:07.975208 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:07.978275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:08.035178 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 15/31] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:08.037238 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d064aba-d2c6-4258-8643-cb3f35d96957 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:08.037630 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d064aba-d2c6-4258-8643-cb3f35d96957 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:08.038117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d064aba-d2c6-4258-8643-cb3f35d96957 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:08.043450 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 16/32] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:08.043450 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d064aba-d2c6-4258-8643-cb3f35d96957 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:08.165092 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d064aba-d2c6-4258-8643-cb3f35d96957 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:08.196147 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 17/33] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jun 30 10:13:07 2022] POST /identity/v3/projects => generated 301 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:13:08.197696 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cc79c-64b2-44ad-97ea-bac8e3585499 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:08.197950 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cc79c-64b2-44ad-97ea-bac8e3585499 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:08.198239 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4cc79c-64b2-44ad-97ea-bac8e3585499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:08.205706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f7f13-218b-43bf-97a1-d887c259e096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:08.206895 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f7f13-218b-43bf-97a1-d887c259e096 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:08.207370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f7f13-218b-43bf-97a1-d887c259e096 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:08.207785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18f7f13-218b-43bf-97a1-d887c259e096 None admin] PATH_INFO: `/v3/users/67e761d82cde42509f040f82a44c86c4` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:08.210268 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f7f13-218b-43bf-97a1-d887c259e096 None admin] RBAC: Authorizing `identity:get_user(user_id=67e761d82cde42509f040f82a44c86c4)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:08.211993 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18f7f13-218b-43bf-97a1-d887c259e096 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:08.213956 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 17/34] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:08 2022] GET /identity/v3/users/67e761d82cde42509f040f82a44c86c4 => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:08.214842 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f7f13-218b-43bf-97a1-d887c259e096 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:08.219482 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-da4cc79c-64b2-44ad-97ea-bac8e3585499 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:08.221141 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-da4cc79c-64b2-44ad-97ea-bac8e3585499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:08.239081 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:08.305392 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 18/35] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:08.306825 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4cc79c-64b2-44ad-97ea-bac8e3585499 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:08.340073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18f7f13-218b-43bf-97a1-d887c259e096 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:08.386999 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86856144-060f-4c28-937e-e79e125498ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:08.388055 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86856144-060f-4c28-937e-e79e125498ea None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:08.388506 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86856144-060f-4c28-937e-e79e125498ea None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:08.389020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86856144-060f-4c28-937e-e79e125498ea None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:08.389923 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86856144-060f-4c28-937e-e79e125498ea None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:08.391924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86856144-060f-4c28-937e-e79e125498ea None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:08.453317 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4cc79c-64b2-44ad-97ea-bac8e3585499 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:08.473440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:08.494368 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9af04-94af-4580-adc4-edf5dedcde63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:08.495181 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9af04-94af-4580-adc4-edf5dedcde63 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:08.495570 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9af04-94af-4580-adc4-edf5dedcde63 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:08.495853 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9af04-94af-4580-adc4-edf5dedcde63 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:08.497074 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9af04-94af-4580-adc4-edf5dedcde63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:08.498827 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9af04-94af-4580-adc4-edf5dedcde63 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:08.500942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 19/36] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:13:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:08.501798 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9af04-94af-4580-adc4-edf5dedcde63 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:08.688315 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 18/37] 10.222.0.47 () {60 vars in 1131 bytes} [Thu Jun 30 10:13:08 2022] POST /identity/v3/roles => generated 225 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:13:08.688315 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86856144-060f-4c28-937e-e79e125498ea None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:08.741557 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9af04-94af-4580-adc4-edf5dedcde63 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:08.778599 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:08.782123 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4553e69c-22e1-4e48-ac1d-34edef1800d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:08.782835 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553e69c-22e1-4e48-ac1d-34edef1800d1 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:08.783122 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553e69c-22e1-4e48-ac1d-34edef1800d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:08.783596 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4553e69c-22e1-4e48-ac1d-34edef1800d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:08.784308 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4553e69c-22e1-4e48-ac1d-34edef1800d1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:08.785771 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4553e69c-22e1-4e48-ac1d-34edef1800d1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:08.802858 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86856144-060f-4c28-937e-e79e125498ea None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:08.839156 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc20f894-fde2-4e3a-86d4-6dd207198de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:08.840076 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20f894-fde2-4e3a-86d4-6dd207198de4 None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:08.840363 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20f894-fde2-4e3a-86d4-6dd207198de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:08.840644 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc20f894-fde2-4e3a-86d4-6dd207198de4 None admin] PATH_INFO: `/v3/domains/default/users/67e761d82cde42509f040f82a44c86c4/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:08.842897 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc20f894-fde2-4e3a-86d4-6dd207198de4 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=67e761d82cde42509f040f82a44c86c4, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:08.844518 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc20f894-fde2-4e3a-86d4-6dd207198de4 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:08.878234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 20/38] 10.222.0.47 () {60 vars in 1131 bytes} [Thu Jun 30 10:13:08 2022] POST /identity/v3/roles => generated 219 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:13:08.879761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517a315-0def-4141-8b4b-04f17f24adee None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:08.880067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517a315-0def-4141-8b4b-04f17f24adee None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:08.881608 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517a315-0def-4141-8b4b-04f17f24adee None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:08.882662 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 21/39] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:08.884058 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f97c97-1ce7-457e-adf7-3dd77a3def9b None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:08.884335 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f97c97-1ce7-457e-adf7-3dd77a3def9b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:08.884686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f97c97-1ce7-457e-adf7-3dd77a3def9b None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:08.886471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 22/40] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:08.889664 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92933ace-6c42-4435-bae4-6f29f809ee10 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:08.889963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92933ace-6c42-4435-bae4-6f29f809ee10 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:08.890254 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92933ace-6c42-4435-bae4-6f29f809ee10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:08.909410 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-92933ace-6c42-4435-bae4-6f29f809ee10 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:08.910679 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-92933ace-6c42-4435-bae4-6f29f809ee10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:08.974652 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 19/41] 10.222.0.47 () {58 vars in 1361 bytes} [Thu Jun 30 10:13:08 2022] PUT /identity/v3/domains/default/users/67e761d82cde42509f040f82a44c86c4/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:13:08.976737 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ef33b-01e0-4045-bd61-62af759f8dc6 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:08.977095 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:08.977504 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ef33b-01e0-4045-bd61-62af759f8dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:08.977990 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674ef33b-01e0-4045-bd61-62af759f8dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:09.001018 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-674ef33b-01e0-4045-bd61-62af759f8dc6 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:09.002593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-674ef33b-01e0-4045-bd61-62af759f8dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:09.032234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 23/42] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:09.058149 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92933ace-6c42-4435-bae4-6f29f809ee10 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:09.096373 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 20/43] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:09.121696 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ef33b-01e0-4045-bd61-62af759f8dc6 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:09.185852 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92933ace-6c42-4435-bae4-6f29f809ee10 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:09.214135 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:09.225252 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b957bd5-205d-4dea-93a1-012cb2dc247f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:09.225944 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b957bd5-205d-4dea-93a1-012cb2dc247f None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:09.226235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b957bd5-205d-4dea-93a1-012cb2dc247f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:09.226513 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b957bd5-205d-4dea-93a1-012cb2dc247f None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:09.227310 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b957bd5-205d-4dea-93a1-012cb2dc247f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:09.228701 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b957bd5-205d-4dea-93a1-012cb2dc247f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:09.251489 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674ef33b-01e0-4045-bd61-62af759f8dc6 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:09.290206 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b237f9fe-05b9-4388-870f-382dfb59f8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:09.291032 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237f9fe-05b9-4388-870f-382dfb59f8fd None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:09.291352 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237f9fe-05b9-4388-870f-382dfb59f8fd None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:09.291636 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237f9fe-05b9-4388-870f-382dfb59f8fd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:09.293132 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b237f9fe-05b9-4388-870f-382dfb59f8fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:09.295025 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b237f9fe-05b9-4388-870f-382dfb59f8fd None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:09.297428 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 21/44] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:13:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:09.302945 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b237f9fe-05b9-4388-870f-382dfb59f8fd None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:09.311748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 24/45] 10.222.0.47 () {60 vars in 1131 bytes} [Thu Jun 30 10:13:09 2022] POST /identity/v3/roles => generated 223 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:13:09.425294 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b237f9fe-05b9-4388-870f-382dfb59f8fd None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:09.465733 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38975d6e-dcc8-4a4f-98d9-93d1f2174db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:09.466431 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38975d6e-dcc8-4a4f-98d9-93d1f2174db9 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:09.466728 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38975d6e-dcc8-4a4f-98d9-93d1f2174db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:09.467040 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38975d6e-dcc8-4a4f-98d9-93d1f2174db9 None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:09.467789 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38975d6e-dcc8-4a4f-98d9-93d1f2174db9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:09.469273 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38975d6e-dcc8-4a4f-98d9-93d1f2174db9 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:09.474282 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:09.542682 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 22/46] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jun 30 10:13:09 2022] POST /identity/v3/projects => generated 312 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:13:11.072584 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e12404-27e4-47f5-b631-0652a848da11 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:11.073133 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e12404-27e4-47f5-b631-0652a848da11 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:11.073133 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e12404-27e4-47f5-b631-0652a848da11 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:11.074537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 25/47] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:11.079862 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b1b7c-549e-4264-a3a4-3447e5b47e00 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:11.080285 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b1b7c-549e-4264-a3a4-3447e5b47e00 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:11.080440 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574b1b7c-549e-4264-a3a4-3447e5b47e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:11.097494 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-574b1b7c-549e-4264-a3a4-3447e5b47e00 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:11.098708 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-574b1b7c-549e-4264-a3a4-3447e5b47e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:11.166234 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 23/48] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:11.189724 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e12404-27e4-47f5-b631-0652a848da11 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:11.226326 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:11.292912 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e12404-27e4-47f5-b631-0652a848da11 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:11.326951 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad0eaf-1a67-4254-8d8b-f0880f6d1fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:11.327709 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad0eaf-1a67-4254-8d8b-f0880f6d1fde None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:11.328002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad0eaf-1a67-4254-8d8b-f0880f6d1fde None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:11.328287 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baad0eaf-1a67-4254-8d8b-f0880f6d1fde None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:11.338864 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baad0eaf-1a67-4254-8d8b-f0880f6d1fde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:11.346548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-baad0eaf-1a67-4254-8d8b-f0880f6d1fde None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:13:11.346548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:11.346548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:11.346548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:11.346548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:11.346548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:11.346548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:11.346548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:11.346548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:11.346548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:11.346548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:11.346548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:11.346548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:11.346548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:11.346548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:11.346548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:11.347432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:11.347432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:11.347432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:11.347432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:11.347432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:11.347432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:11.347432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:11.347432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:11.347432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:11.347432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:11.347432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:11.347432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:11.347432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:11.347432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:11.347432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:11.347432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:11.348190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:11.348190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:11.348190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:11.348190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:11.348190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:11.348190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:11.348190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:11.348190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:11.348190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:11.348190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:11.348190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:13:11.348190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:13:11.348190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 26/49] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:13:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:11.355573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b1b7c-549e-4264-a3a4-3447e5b47e00 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:11.455008 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574b1b7c-549e-4264-a3a4-3447e5b47e00 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:11.577772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:11.594197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe9118-6e6b-43ea-939a-1e02a2d3e5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:11.594907 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe9118-6e6b-43ea-939a-1e02a2d3e5d5 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:11.595194 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe9118-6e6b-43ea-939a-1e02a2d3e5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:11.595508 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe9118-6e6b-43ea-939a-1e02a2d3e5d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:11.596265 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fe9118-6e6b-43ea-939a-1e02a2d3e5d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:11.596577 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fe9118-6e6b-43ea-939a-1e02a2d3e5d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:11.597963 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fe9118-6e6b-43ea-939a-1e02a2d3e5d5 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:11.606804 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 24/50] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:11.612347 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad0eaf-1a67-4254-8d8b-f0880f6d1fde None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:11.704212 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9fc70-9cdf-4e0e-9b69-829be5f56945 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:11.704619 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9fc70-9cdf-4e0e-9b69-829be5f56945 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:11.704971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf9fc70-9cdf-4e0e-9b69-829be5f56945 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:11.706036 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:11.706538 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 25/51] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:13:11.711736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f93fa-1ffe-44b7-8112-e34a2d5506a4 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:11.712132 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f93fa-1ffe-44b7-8112-e34a2d5506a4 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:11.712471 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7f93fa-1ffe-44b7-8112-e34a2d5506a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:11.712694 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baad0eaf-1a67-4254-8d8b-f0880f6d1fde None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:11.731639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-7a7f93fa-1ffe-44b7-8112-e34a2d5506a4 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:11.733154 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-7a7f93fa-1ffe-44b7-8112-e34a2d5506a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:11.748359 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a91e7-2029-4eb5-aa17-d138da820fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:11.749247 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a91e7-2029-4eb5-aa17-d138da820fce None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:11.749531 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a91e7-2029-4eb5-aa17-d138da820fce None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:11.749807 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8a91e7-2029-4eb5-aa17-d138da820fce None admin] PATH_INFO: `/v3/users/67e761d82cde42509f040f82a44c86c4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:11.751013 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8a91e7-2029-4eb5-aa17-d138da820fce None admin] RBAC: Authorizing `identity:get_user(user_id=67e761d82cde42509f040f82a44c86c4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:11.752522 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8a91e7-2029-4eb5-aa17-d138da820fce None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:11.754356 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 27/52] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:11 2022] GET /identity/v3/users/67e761d82cde42509f040f82a44c86c4 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:11.760326 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a91e7-2029-4eb5-aa17-d138da820fce None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:11.812367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 26/53] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 10:13:11.835490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f93fa-1ffe-44b7-8112-e34a2d5506a4 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:11.858544 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8a91e7-2029-4eb5-aa17-d138da820fce None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:11.894342 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c1ff6-1dea-4a21-84ed-5875a9d25eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:11.895135 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c1ff6-1dea-4a21-84ed-5875a9d25eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:11.895454 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c1ff6-1dea-4a21-84ed-5875a9d25eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:11.895789 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c1ff6-1dea-4a21-84ed-5875a9d25eb7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:11.897238 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c1ff6-1dea-4a21-84ed-5875a9d25eb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:11.898704 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c1ff6-1dea-4a21-84ed-5875a9d25eb7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:11.900365 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 28/54] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:13:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:11.905933 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c1ff6-1dea-4a21-84ed-5875a9d25eb7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:11.935590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7f93fa-1ffe-44b7-8112-e34a2d5506a4 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:11.936103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:11.949204 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:11.972864 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41e9ad-784b-4d7c-b05f-44db5c2859b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:11.973669 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41e9ad-784b-4d7c-b05f-44db5c2859b5 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:11.974093 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41e9ad-784b-4d7c-b05f-44db5c2859b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:11.974386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a41e9ad-784b-4d7c-b05f-44db5c2859b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:11.975627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41e9ad-784b-4d7c-b05f-44db5c2859b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:11.977094 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a41e9ad-784b-4d7c-b05f-44db5c2859b5 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:11.978725 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 27/55] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:13:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:11.988151 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41e9ad-784b-4d7c-b05f-44db5c2859b5 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:12.013632 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c1ff6-1dea-4a21-84ed-5875a9d25eb7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:12.047864 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73abef2-f441-4558-b567-8384fd0a5b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:12.048520 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73abef2-f441-4558-b567-8384fd0a5b81 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:12.048809 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73abef2-f441-4558-b567-8384fd0a5b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:12.049092 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73abef2-f441-4558-b567-8384fd0a5b81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:12.049925 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73abef2-f441-4558-b567-8384fd0a5b81 None admin] RBAC: Adding query filter params (role.id=e63bda042e104a4a93fba15ffcfe1183, scope.domain.id=default, user.id=67e761d82cde42509f040f82a44c86c4) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:12.050228 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73abef2-f441-4558-b567-8384fd0a5b81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:12.051728 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73abef2-f441-4558-b567-8384fd0a5b81 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:12.058585 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 29/56] 10.222.0.47 () {56 vars in 1322 bytes} [Thu Jun 30 10:13:11 2022] GET /identity/v3/role_assignments?user.id=67e761d82cde42509f040f82a44c86c4&scope.domain.id=default&role.id=e63bda042e104a4a93fba15ffcfe1183 => generated 530 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:12.088052 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a41e9ad-784b-4d7c-b05f-44db5c2859b5 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:12.125881 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6317d3-9f0b-46dc-aeea-56a6511e4a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:12.126577 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6317d3-9f0b-46dc-aeea-56a6511e4a07 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:12.126870 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6317d3-9f0b-46dc-aeea-56a6511e4a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:12.127154 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6317d3-9f0b-46dc-aeea-56a6511e4a07 None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:12.127931 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6317d3-9f0b-46dc-aeea-56a6511e4a07 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:12.129427 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6317d3-9f0b-46dc-aeea-56a6511e4a07 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:12.209270 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 28/57] 10.222.0.47 () {60 vars in 1140 bytes} [Thu Jun 30 10:13:11 2022] POST /identity/v3/projects => generated 298 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:13:14.242177 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24698de-5ac7-4f18-aca4-cac494431314 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:14.242705 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24698de-5ac7-4f18-aca4-cac494431314 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:14.242787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24698de-5ac7-4f18-aca4-cac494431314 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:14.244159 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 30/58] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:14.249295 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608646b-01fe-48b7-828d-6fe3486f8f8a None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:14.249715 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608646b-01fe-48b7-828d-6fe3486f8f8a None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:14.249865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608646b-01fe-48b7-828d-6fe3486f8f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:14.266389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-a608646b-01fe-48b7-828d-6fe3486f8f8a None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:14.267621 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-a608646b-01fe-48b7-828d-6fe3486f8f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:14.292141 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:14.334198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 29/59] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:14.362123 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24698de-5ac7-4f18-aca4-cac494431314 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:14.454172 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:14.462114 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24698de-5ac7-4f18-aca4-cac494431314 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:14.494298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060b903-35bb-4950-902f-121e8bce1a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:14.495020 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060b903-35bb-4950-902f-121e8bce1a9c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:14.495336 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060b903-35bb-4950-902f-121e8bce1a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:14.495620 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060b903-35bb-4950-902f-121e8bce1a9c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:14.496798 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060b903-35bb-4950-902f-121e8bce1a9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:14.498273 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060b903-35bb-4950-902f-121e8bce1a9c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:14.499938 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 31/60] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:13:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:14.506070 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608646b-01fe-48b7-828d-6fe3486f8f8a None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:14.604604 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608646b-01fe-48b7-828d-6fe3486f8f8a None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:14.637204 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b4b66-7b77-4956-810d-6eec13a19a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:14.637874 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b4b66-7b77-4956-810d-6eec13a19a0a None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:14.638165 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b4b66-7b77-4956-810d-6eec13a19a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:14.638456 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b4b66-7b77-4956-810d-6eec13a19a0a None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:14.639184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b4b66-7b77-4956-810d-6eec13a19a0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:14.640671 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b4b66-7b77-4956-810d-6eec13a19a0a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:14.734550 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 30/61] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jun 30 10:13:14 2022] POST /identity/v3/users => generated 275 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:13:17.788896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c8506-d798-4661-8028-c890daf60879 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:17.788896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c8506-d798-4661-8028-c890daf60879 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:17.788896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c8506-d798-4661-8028-c890daf60879 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:17.789887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 32/62] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:17.795459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3ef4c-74d3-4369-b5cd-3c686b8ae868 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:17.796282 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3ef4c-74d3-4369-b5cd-3c686b8ae868 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:17.796802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d3ef4c-74d3-4369-b5cd-3c686b8ae868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:17.820266 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-b0d3ef4c-74d3-4369-b5cd-3c686b8ae868 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:17.821841 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-b0d3ef4c-74d3-4369-b5cd-3c686b8ae868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:17.839198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:17.916242 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 31/63] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:17.954869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c8506-d798-4661-8028-c890daf60879 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:18.001516 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13583ec8-e9bc-43ba-99e3-f9692ce46db1 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:18.001842 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13583ec8-e9bc-43ba-99e3-f9692ce46db1 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:18.002095 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13583ec8-e9bc-43ba-99e3-f9692ce46db1 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:18.003519 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 32/64] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:13:18.009061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bc9da-c282-4ecb-a148-448f5ea20a9f None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:18.009597 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bc9da-c282-4ecb-a148-448f5ea20a9f None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:18.010086 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bc9da-c282-4ecb-a148-448f5ea20a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:18.027373 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-2b4bc9da-c282-4ecb-a148-448f5ea20a9f None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:18.028895 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-2b4bc9da-c282-4ecb-a148-448f5ea20a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:18.108210 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c8506-d798-4661-8028-c890daf60879 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:18.156242 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520c34c-e047-4d4a-846c-be05f256bae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:18.157055 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520c34c-e047-4d4a-846c-be05f256bae4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:18.157344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520c34c-e047-4d4a-846c-be05f256bae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:18.157627 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b520c34c-e047-4d4a-846c-be05f256bae4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:18.160299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 33/65] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 10:13:18.179099 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b520c34c-e047-4d4a-846c-be05f256bae4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:18.184621 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4bc9da-c282-4ecb-a148-448f5ea20a9f None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:18.187353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-b520c34c-e047-4d4a-846c-be05f256bae4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:18.187353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:18.187353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:18.187353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:18.187353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:18.187353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:18.187353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:18.187353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:18.187353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:18.187353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:18.187353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:18.187353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:18.187353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:18.187353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:18.187353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:18.187353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:18.188194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:18.188194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:18.188194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:18.188194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:18.188194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:18.188194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:18.188194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:18.188194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:18.188194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:18.188194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:18.188194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:18.188194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:18.188194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:18.188194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:18.188194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:18.188194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:18.188949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:18.188949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:18.188949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:18.188949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:18.188949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:18.188949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:18.188949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:18.188949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:18.188949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:18.188949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:18.188949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:18.188949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:13:18.188949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 33/66] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:18.196618 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520c34c-e047-4d4a-846c-be05f256bae4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:18.240617 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:18.276865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:18.303503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4bc9da-c282-4ecb-a148-448f5ea20a9f None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:18.348417 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013969d-8c03-486a-9809-12e803ccc211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:18.350748 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013969d-8c03-486a-9809-12e803ccc211 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:18.351254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013969d-8c03-486a-9809-12e803ccc211 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:18.351764 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013969d-8c03-486a-9809-12e803ccc211 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:18.366425 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e013969d-8c03-486a-9809-12e803ccc211 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:18.375457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-e013969d-8c03-486a-9809-12e803ccc211 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:13:18.375457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:18.375457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:18.375457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:18.375457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:18.375457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:18.375457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:18.375457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:18.375457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:18.375457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:18.375457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:18.375457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:18.375457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:18.375457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:18.375457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:18.375457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:18.376323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:18.376323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:18.376323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:18.376323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:18.376323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:18.376323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:18.376323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:18.376323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:18.376323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:18.376323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:18.376323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:18.376323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:18.376323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:18.376323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:18.376323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:18.376323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:18.377146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:18.377146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:18.377146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:18.377146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:18.377146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:18.377146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:18.377146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:18.377146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:18.377146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:18.377146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:18.377146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:13:18.377146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:13:18.379309 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 34/67] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:13:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:13:18.398136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013969d-8c03-486a-9809-12e803ccc211 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:18.414617 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b520c34c-e047-4d4a-846c-be05f256bae4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:18.476870 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c33197-e93c-4227-bd78-527c1ecc1c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:18.478305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c33197-e93c-4227-bd78-527c1ecc1c97 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:18.479129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c33197-e93c-4227-bd78-527c1ecc1c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:18.479644 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c33197-e93c-4227-bd78-527c1ecc1c97 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:18.481079 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c33197-e93c-4227-bd78-527c1ecc1c97 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:18.481432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c33197-e93c-4227-bd78-527c1ecc1c97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:18.483175 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c33197-e93c-4227-bd78-527c1ecc1c97 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:18.498671 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 34/68] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:13:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:18.498671 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822f06b-6892-4093-a837-cce08130aaee None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:18.498671 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822f06b-6892-4093-a837-cce08130aaee None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:18.498671 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c822f06b-6892-4093-a837-cce08130aaee None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:18.498671 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 35/69] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:18.499422 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee8064-ac67-4bf7-9e1a-2422ed36a096 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:18.499936 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee8064-ac67-4bf7-9e1a-2422ed36a096 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:18.500439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ee8064-ac67-4bf7-9e1a-2422ed36a096 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:18.502320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 36/70] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:18.503259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee8064-ac67-4bf7-9e1a-2422ed36a096 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:18.530633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e013969d-8c03-486a-9809-12e803ccc211 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:18.571024 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b72f13b-7b9f-4e86-8345-8dc0766e54b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:18.572235 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b72f13b-7b9f-4e86-8345-8dc0766e54b5 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:18.573022 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b72f13b-7b9f-4e86-8345-8dc0766e54b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:18.573495 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b72f13b-7b9f-4e86-8345-8dc0766e54b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:18.574451 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b72f13b-7b9f-4e86-8345-8dc0766e54b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:18.574951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b72f13b-7b9f-4e86-8345-8dc0766e54b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:18.576667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b72f13b-7b9f-4e86-8345-8dc0766e54b5 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:18.579628 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:18.593471 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 35/71] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:18.593471 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba5e31-5a5c-4655-a2e2-2078c572d2fa None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:18.593471 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba5e31-5a5c-4655-a2e2-2078c572d2fa None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:18.593471 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ba5e31-5a5c-4655-a2e2-2078c572d2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:18.598122 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:18.610985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-99ba5e31-5a5c-4655-a2e2-2078c572d2fa None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:18.612246 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-99ba5e31-5a5c-4655-a2e2-2078c572d2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:18.619240 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ee8064-ac67-4bf7-9e1a-2422ed36a096 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:18.659444 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecac8678-f43d-4955-a5d0-a7944df6eb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:18.660429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac8678-f43d-4955-a5d0-a7944df6eb9b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:18.660721 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac8678-f43d-4955-a5d0-a7944df6eb9b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:18.661075 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecac8678-f43d-4955-a5d0-a7944df6eb9b None admin] PATH_INFO: `/v3/users/db5cbfa9be364e0d9c75eac6090e7967` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:18.679993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecac8678-f43d-4955-a5d0-a7944df6eb9b None admin] RBAC: Authorizing `identity:get_user(user_id=db5cbfa9be364e0d9c75eac6090e7967)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:18.682329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecac8678-f43d-4955-a5d0-a7944df6eb9b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:18.688454 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 37/72] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:18 2022] GET /identity/v3/users/db5cbfa9be364e0d9c75eac6090e7967 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:18.688454 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae309e76-2bcd-471b-b77b-4cd26bfd77b6 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:18.688454 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae309e76-2bcd-471b-b77b-4cd26bfd77b6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:18.688454 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae309e76-2bcd-471b-b77b-4cd26bfd77b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:18.711291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 36/73] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:18.711291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba5e31-5a5c-4655-a2e2-2078c572d2fa None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:18.712872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ae309e76-2bcd-471b-b77b-4cd26bfd77b6 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:18.714160 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ae309e76-2bcd-471b-b77b-4cd26bfd77b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:18.785297 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 38/74] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:18.787288 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae309e76-2bcd-471b-b77b-4cd26bfd77b6 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:18.816349 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ba5e31-5a5c-4655-a2e2-2078c572d2fa None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:18.845002 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:18.853622 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e877e-beaa-456a-8ba6-0d80c3a9f7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:18.854510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e877e-beaa-456a-8ba6-0d80c3a9f7eb None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:18.854802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e877e-beaa-456a-8ba6-0d80c3a9f7eb None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:18.855086 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e877e-beaa-456a-8ba6-0d80c3a9f7eb None admin] PATH_INFO: `/v3/users/67e761d82cde42509f040f82a44c86c4` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:18.857862 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e877e-beaa-456a-8ba6-0d80c3a9f7eb None admin] RBAC: Authorizing `identity:get_user(user_id=67e761d82cde42509f040f82a44c86c4)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:18.857987 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:18.858435 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37e877e-beaa-456a-8ba6-0d80c3a9f7eb None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:18.859481 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 37/75] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:18 2022] GET /identity/v3/users/67e761d82cde42509f040f82a44c86c4 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:18.860265 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e877e-beaa-456a-8ba6-0d80c3a9f7eb None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:18.887666 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae309e76-2bcd-471b-b77b-4cd26bfd77b6 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:18.923046 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1d6a4-8b0a-418a-83ed-ef79a267f397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:18.923738 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1d6a4-8b0a-418a-83ed-ef79a267f397 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:18.924034 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1d6a4-8b0a-418a-83ed-ef79a267f397 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:18.924320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d1d6a4-8b0a-418a-83ed-ef79a267f397 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:18.967064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e877e-beaa-456a-8ba6-0d80c3a9f7eb None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:18.994134 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d1d6a4-8b0a-418a-83ed-ef79a267f397 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:18.995663 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d1d6a4-8b0a-418a-83ed-ef79a267f397 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:19.002377 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344146e8-80fa-44c2-a468-6fecfd6e8f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:19.003599 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344146e8-80fa-44c2-a468-6fecfd6e8f18 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:19.003890 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344146e8-80fa-44c2-a468-6fecfd6e8f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:19.004171 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344146e8-80fa-44c2-a468-6fecfd6e8f18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:19.017681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344146e8-80fa-44c2-a468-6fecfd6e8f18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:19.030110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-344146e8-80fa-44c2-a468-6fecfd6e8f18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:19.030110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:19.030110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:19.030110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:19.030110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:19.030110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:19.030110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:19.030110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:19.030110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:19.030110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:19.030110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:19.030110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:19.030110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:19.030110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:19.030110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:19.030110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:19.032009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:19.032009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:19.032009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:19.032009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:19.032009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:19.032009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:19.032009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:19.032009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:19.032009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:19.032009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:19.032009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:19.032009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:19.032009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:19.032009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:19.032009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:19.032009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:19.032785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:19.032785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:19.032785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:19.032785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:19.032785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:19.032785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:19.032785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:19.032785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:19.032785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:19.032785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:19.032785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:19.032785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:13:19.032785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 38/76] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:19.032785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f2d05-40fa-41ff-afc5-dd75860cee34 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:19.033469 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f2d05-40fa-41ff-afc5-dd75860cee34 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:19.033469 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f2d05-40fa-41ff-afc5-dd75860cee34 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:19.035352 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 39/77] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:19.035352 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f2d05-40fa-41ff-afc5-dd75860cee34 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:19.070901 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:19.098014 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:19.106043 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 39/78] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:13:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:13:19.107518 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1d6a4-8b0a-418a-83ed-ef79a267f397 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:19.147523 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f2d05-40fa-41ff-afc5-dd75860cee34 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:19.179700 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a38117-05bf-4b80-b709-9f33f9f3dd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:19.180507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a38117-05bf-4b80-b709-9f33f9f3dd50 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:19.180793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a38117-05bf-4b80-b709-9f33f9f3dd50 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:19.181071 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a38117-05bf-4b80-b709-9f33f9f3dd50 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:19.191166 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a38117-05bf-4b80-b709-9f33f9f3dd50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:19.199996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-a0a38117-05bf-4b80-b709-9f33f9f3dd50 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 10:13:19.199996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:19.199996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:19.199996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:19.199996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:19.199996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:19.199996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:19.199996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:19.199996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:19.199996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:19.199996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:19.199996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:19.199996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:19.199996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:19.199996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:19.199996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:19.200852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:19.200852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:19.200852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:19.200852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:19.200852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:19.200852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:19.200852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:19.200852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:19.200852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:19.200852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:19.200852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:19.200852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:19.200852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:19.200852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:19.200852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:19.200852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:19.201617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:19.201617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:19.201617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:19.201617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:19.201617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:19.201617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:19.201617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:19.201617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:19.201617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:19.201617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:19.201617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 10:13:19.201617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:13:19.201617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 40/79] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jun 30 10:13:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:19.202267 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a38117-05bf-4b80-b709-9f33f9f3dd50 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:19.209513 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d1d6a4-8b0a-418a-83ed-ef79a267f397 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:19.242873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d1c088-6568-4bd5-9f8e-6efca3f49ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:19.243571 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1c088-6568-4bd5-9f8e-6efca3f49ebb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:19.243861 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1c088-6568-4bd5-9f8e-6efca3f49ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:19.244139 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1c088-6568-4bd5-9f8e-6efca3f49ebb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:19.284269 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:19.305977 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a38117-05bf-4b80-b709-9f33f9f3dd50 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:19.311131 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d1c088-6568-4bd5-9f8e-6efca3f49ebb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:19.312691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d1c088-6568-4bd5-9f8e-6efca3f49ebb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:19.322470 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:19.339395 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3dafc9-938c-4637-a155-12717f86f676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:19.340072 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3dafc9-938c-4637-a155-12717f86f676 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:19.340362 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3dafc9-938c-4637-a155-12717f86f676 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:19.340650 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3dafc9-938c-4637-a155-12717f86f676 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:19.341405 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3dafc9-938c-4637-a155-12717f86f676 None admin] RBAC: Adding query filter params (name=member) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:19.341718 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3dafc9-938c-4637-a155-12717f86f676 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:19.343102 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3dafc9-938c-4637-a155-12717f86f676 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:19.351326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 41/80] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:13:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:19.352537 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f8707-b893-4e7d-ad34-4498a5f590f5 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:19.352833 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f8707-b893-4e7d-ad34-4498a5f590f5 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:19.353124 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f8707-b893-4e7d-ad34-4498a5f590f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:19.369603 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-990f8707-b893-4e7d-ad34-4498a5f590f5 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:19.370814 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-990f8707-b893-4e7d-ad34-4498a5f590f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:19.428370 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 40/81] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:13:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:13:19.429233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d1c088-6568-4bd5-9f8e-6efca3f49ebb None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:19.444795 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 42/82] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:19.445568 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f8707-b893-4e7d-ad34-4498a5f590f5 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:19.504120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:19.534703 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d1c088-6568-4bd5-9f8e-6efca3f49ebb None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:19.555887 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f8707-b893-4e7d-ad34-4498a5f590f5 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:19.565016 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:19.568714 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bc70c5-5cbc-4409-9085-88c22ba2a24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:19.569570 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc70c5-5cbc-4409-9085-88c22ba2a24b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:19.569858 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc70c5-5cbc-4409-9085-88c22ba2a24b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:19.570134 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bc70c5-5cbc-4409-9085-88c22ba2a24b None admin] PATH_INFO: `/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:19.571347 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bc70c5-5cbc-4409-9085-88c22ba2a24b None admin] RBAC: Authorizing `identity:get_project(project_id=47885b7cd44548ce92d8cfb4b55c8a97)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:19.572861 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bc70c5-5cbc-4409-9085-88c22ba2a24b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:19.574581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 41/83] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:19 2022] GET /identity/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:19.575370 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bc70c5-5cbc-4409-9085-88c22ba2a24b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:19.590160 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cad3b-661d-4367-9cc5-a0ef7f092ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:19.590840 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cad3b-661d-4367-9cc5-a0ef7f092ba1 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:19.591139 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cad3b-661d-4367-9cc5-a0ef7f092ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:19.591436 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19cad3b-661d-4367-9cc5-a0ef7f092ba1 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:19.592185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19cad3b-661d-4367-9cc5-a0ef7f092ba1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:19.592492 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19cad3b-661d-4367-9cc5-a0ef7f092ba1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:19.593875 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19cad3b-661d-4367-9cc5-a0ef7f092ba1 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:19.603165 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 43/84] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:13:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:19.604157 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cad3b-661d-4367-9cc5-a0ef7f092ba1 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:19.678774 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bc70c5-5cbc-4409-9085-88c22ba2a24b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:19.703109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19cad3b-661d-4367-9cc5-a0ef7f092ba1 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:19.712031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0cd09-c873-406b-8108-7d6a06ac50a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:19.712899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0cd09-c873-406b-8108-7d6a06ac50a3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:19.713192 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0cd09-c873-406b-8108-7d6a06ac50a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:19.713471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b0cd09-c873-406b-8108-7d6a06ac50a3 None admin] PATH_INFO: `/v3/users/db5cbfa9be364e0d9c75eac6090e7967` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:19.714676 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0cd09-c873-406b-8108-7d6a06ac50a3 None admin] RBAC: Authorizing `identity:get_user(user_id=db5cbfa9be364e0d9c75eac6090e7967)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:19.716176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b0cd09-c873-406b-8108-7d6a06ac50a3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:19.717802 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 42/85] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:19 2022] GET /identity/v3/users/db5cbfa9be364e0d9c75eac6090e7967 => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:19.718554 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0cd09-c873-406b-8108-7d6a06ac50a3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:19.731194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:19.737068 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf020b16-6b57-47ea-a4bc-5084884fb570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:19.737940 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf020b16-6b57-47ea-a4bc-5084884fb570 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:19.738229 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf020b16-6b57-47ea-a4bc-5084884fb570 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:19.738512 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf020b16-6b57-47ea-a4bc-5084884fb570 None admin] PATH_INFO: `/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:19.739886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf020b16-6b57-47ea-a4bc-5084884fb570 None admin] RBAC: Authorizing `identity:get_project(project_id=47885b7cd44548ce92d8cfb4b55c8a97)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:19.741342 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf020b16-6b57-47ea-a4bc-5084884fb570 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:19.747930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 44/86] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:19 2022] GET /identity/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:19.747930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf020b16-6b57-47ea-a4bc-5084884fb570 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:19.788021 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:19.821348 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b0cd09-c873-406b-8108-7d6a06ac50a3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:19.847593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf020b16-6b57-47ea-a4bc-5084884fb570 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:19.854393 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66941838-afdc-43a9-9eb6-591d8598634f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:19.855191 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66941838-afdc-43a9-9eb6-591d8598634f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:19.855512 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66941838-afdc-43a9-9eb6-591d8598634f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:19.855796 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66941838-afdc-43a9-9eb6-591d8598634f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:19.856992 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66941838-afdc-43a9-9eb6-591d8598634f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:19.858448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66941838-afdc-43a9-9eb6-591d8598634f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:19.860118 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 43/87] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:13:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:19.860888 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66941838-afdc-43a9-9eb6-591d8598634f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:19.886294 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedead7e-7f7e-4d72-bc71-f120fb6715e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:19.886954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedead7e-7f7e-4d72-bc71-f120fb6715e4 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:19.887257 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedead7e-7f7e-4d72-bc71-f120fb6715e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:19.887557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedead7e-7f7e-4d72-bc71-f120fb6715e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:19.888393 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedead7e-7f7e-4d72-bc71-f120fb6715e4 None admin] RBAC: Adding query filter params (role.id=31034f95ef88402296c753c2c259f717, scope.project.id=47885b7cd44548ce92d8cfb4b55c8a97, user.id=db5cbfa9be364e0d9c75eac6090e7967) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:19.888706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedead7e-7f7e-4d72-bc71-f120fb6715e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:19.890151 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedead7e-7f7e-4d72-bc71-f120fb6715e4 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:19.898482 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 45/88] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:13:19 2022] GET /identity/v3/role_assignments?user.id=db5cbfa9be364e0d9c75eac6090e7967&scope.project.id=47885b7cd44548ce92d8cfb4b55c8a97&role.id=31034f95ef88402296c753c2c259f717 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:19.899296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedead7e-7f7e-4d72-bc71-f120fb6715e4 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:19.941784 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:19.970337 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66941838-afdc-43a9-9eb6-591d8598634f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:20.009400 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2961a2-ccd1-444a-877a-1bc1d50c5284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:20.010309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2961a2-ccd1-444a-877a-1bc1d50c5284 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:20.010757 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2961a2-ccd1-444a-877a-1bc1d50c5284 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:20.011044 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2961a2-ccd1-444a-877a-1bc1d50c5284 None admin] PATH_INFO: `/v3/users/db5cbfa9be364e0d9c75eac6090e7967` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:20.012369 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2961a2-ccd1-444a-877a-1bc1d50c5284 None admin] RBAC: Authorizing `identity:get_user(user_id=db5cbfa9be364e0d9c75eac6090e7967)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:20.013971 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d2961a2-ccd1-444a-877a-1bc1d50c5284 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:20.020636 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 44/89] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:19 2022] GET /identity/v3/users/db5cbfa9be364e0d9c75eac6090e7967 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:20.020636 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2961a2-ccd1-444a-877a-1bc1d50c5284 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:20.029235 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedead7e-7f7e-4d72-bc71-f120fb6715e4 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:20.030912 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:20.068243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb07620-84f5-42fb-8580-80d0161b8cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:20.068920 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb07620-84f5-42fb-8580-80d0161b8cce None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:20.069213 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb07620-84f5-42fb-8580-80d0161b8cce None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:20.069498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb07620-84f5-42fb-8580-80d0161b8cce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:20.135092 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2961a2-ccd1-444a-877a-1bc1d50c5284 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:20.140285 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb07620-84f5-42fb-8580-80d0161b8cce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:20.141809 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb07620-84f5-42fb-8580-80d0161b8cce None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:20.171855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93028f-32ac-4a13-832c-32575e2b2d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:20.172528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93028f-32ac-4a13-832c-32575e2b2d4b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:20.172817 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93028f-32ac-4a13-832c-32575e2b2d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:20.173101 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93028f-32ac-4a13-832c-32575e2b2d4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:20.173945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93028f-32ac-4a13-832c-32575e2b2d4b None admin] RBAC: Adding query filter params (role.id=e63bda042e104a4a93fba15ffcfe1183, scope.project.id=47885b7cd44548ce92d8cfb4b55c8a97, user.id=67e761d82cde42509f040f82a44c86c4) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:20.174264 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93028f-32ac-4a13-832c-32575e2b2d4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:20.175749 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93028f-32ac-4a13-832c-32575e2b2d4b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:20.182844 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 45/90] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:13:20 2022] GET /identity/v3/role_assignments?user.id=67e761d82cde42509f040f82a44c86c4&scope.project.id=47885b7cd44548ce92d8cfb4b55c8a97&role.id=e63bda042e104a4a93fba15ffcfe1183 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:20.183761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93028f-32ac-4a13-832c-32575e2b2d4b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:20.201218 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:20.279334 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 46/91] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:13:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:13:20.279334 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb07620-84f5-42fb-8580-80d0161b8cce None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:20.281324 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:20.329732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93028f-32ac-4a13-832c-32575e2b2d4b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:20.369488 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc411760-63ac-4b51-b85c-dffce00f0e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:20.370795 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc411760-63ac-4b51-b85c-dffce00f0e15 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:20.371224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc411760-63ac-4b51-b85c-dffce00f0e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:20.371815 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc411760-63ac-4b51-b85c-dffce00f0e15 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:20.373480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc411760-63ac-4b51-b85c-dffce00f0e15 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:20.375097 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc411760-63ac-4b51-b85c-dffce00f0e15 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:20.405127 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb07620-84f5-42fb-8580-80d0161b8cce None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:20.441717 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a522f4b-f2c1-45ca-971a-d5bcf38aca07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:20.442939 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a522f4b-f2c1-45ca-971a-d5bcf38aca07 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:20.443244 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a522f4b-f2c1-45ca-971a-d5bcf38aca07 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:20.443795 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a522f4b-f2c1-45ca-971a-d5bcf38aca07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:20.471554 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 46/92] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jun 30 10:13:20 2022] POST /identity/v3/users => generated 289 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:13:20.474392 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc411760-63ac-4b51-b85c-dffce00f0e15 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:20.561210 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:20.610674 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc411760-63ac-4b51-b85c-dffce00f0e15 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:20.624528 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a522f4b-f2c1-45ca-971a-d5bcf38aca07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:20.625908 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a522f4b-f2c1-45ca-971a-d5bcf38aca07 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:20.630483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:20.656701 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312152bf-7d5d-4694-acf8-1ac4a46c1362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:20.657280 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312152bf-7d5d-4694-acf8-1ac4a46c1362 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:20.657567 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312152bf-7d5d-4694-acf8-1ac4a46c1362 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:20.657847 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312152bf-7d5d-4694-acf8-1ac4a46c1362 None admin] PATH_INFO: `/v3/projects/ce606c0a9d6547819fdbd91ffce253f6` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:20.659771 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312152bf-7d5d-4694-acf8-1ac4a46c1362 None admin] RBAC: Authorizing `identity:get_project(project_id=ce606c0a9d6547819fdbd91ffce253f6)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:20.662436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312152bf-7d5d-4694-acf8-1ac4a46c1362 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:20.664517 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 47/93] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:20 2022] GET /identity/v3/projects/ce606c0a9d6547819fdbd91ffce253f6 => generated 312 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:20.670814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312152bf-7d5d-4694-acf8-1ac4a46c1362 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:20.763566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 47/94] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:13:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:13:20.770629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a522f4b-f2c1-45ca-971a-d5bcf38aca07 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:20.794081 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312152bf-7d5d-4694-acf8-1ac4a46c1362 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:20.828693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:20.835258 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ab1d0-a882-49fa-bea2-ebafae7c73e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:20.836029 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ab1d0-a882-49fa-bea2-ebafae7c73e5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:20.836583 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ab1d0-a882-49fa-bea2-ebafae7c73e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:20.836687 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764ab1d0-a882-49fa-bea2-ebafae7c73e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:20.837625 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ab1d0-a882-49fa-bea2-ebafae7c73e5 None admin] RBAC: Adding query filter params (role.id=31034f95ef88402296c753c2c259f717, scope.project.id=ce606c0a9d6547819fdbd91ffce253f6, user.id=db5cbfa9be364e0d9c75eac6090e7967) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:20.837987 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ab1d0-a882-49fa-bea2-ebafae7c73e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:20.841020 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764ab1d0-a882-49fa-bea2-ebafae7c73e5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:20.845776 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 48/95] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:13:20 2022] GET /identity/v3/role_assignments?user.id=db5cbfa9be364e0d9c75eac6090e7967&scope.project.id=ce606c0a9d6547819fdbd91ffce253f6&role.id=31034f95ef88402296c753c2c259f717 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:20.904272 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a522f4b-f2c1-45ca-971a-d5bcf38aca07 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:20.906556 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:20.957122 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642cb16b-056e-4d90-a225-0b4f687163d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:20.958732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cb16b-056e-4d90-a225-0b4f687163d3 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:20.959317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cb16b-056e-4d90-a225-0b4f687163d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:20.959402 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642cb16b-056e-4d90-a225-0b4f687163d3 None admin] PATH_INFO: `/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:20.960720 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642cb16b-056e-4d90-a225-0b4f687163d3 None admin] RBAC: Authorizing `identity:get_project(project_id=47885b7cd44548ce92d8cfb4b55c8a97)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:20.962202 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642cb16b-056e-4d90-a225-0b4f687163d3 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:20.964176 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 48/96] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:20 2022] GET /identity/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:20.970178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ab1d0-a882-49fa-bea2-ebafae7c73e5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:21.092372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764ab1d0-a882-49fa-bea2-ebafae7c73e5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:21.132358 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d76ae-5203-4cb3-8e7d-17d9dc2b74a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:21.133043 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d76ae-5203-4cb3-8e7d-17d9dc2b74a9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:21.133337 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d76ae-5203-4cb3-8e7d-17d9dc2b74a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:21.133626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d76ae-5203-4cb3-8e7d-17d9dc2b74a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:21.134489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08d76ae-5203-4cb3-8e7d-17d9dc2b74a9 None admin] RBAC: Adding query filter params (role.id=284d1573e06d47eba0196afbe92d4d7d, scope.project.id=47885b7cd44548ce92d8cfb4b55c8a97, user.id=db5cbfa9be364e0d9c75eac6090e7967) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:21.134812 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08d76ae-5203-4cb3-8e7d-17d9dc2b74a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:21.136330 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08d76ae-5203-4cb3-8e7d-17d9dc2b74a9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:21.142654 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 49/97] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:13:20 2022] GET /identity/v3/role_assignments?user.id=db5cbfa9be364e0d9c75eac6090e7967&scope.project.id=47885b7cd44548ce92d8cfb4b55c8a97&role.id=284d1573e06d47eba0196afbe92d4d7d => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:22.403092 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e764e-569b-4399-ba04-00c5fa110e89 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:22.403634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e764e-569b-4399-ba04-00c5fa110e89 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:22.403707 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3e764e-569b-4399-ba04-00c5fa110e89 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:22.405300 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 49/98] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:22.410383 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41053103-e5c0-4f72-8af9-8d16e87ffa0c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:22.411244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41053103-e5c0-4f72-8af9-8d16e87ffa0c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:22.411761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41053103-e5c0-4f72-8af9-8d16e87ffa0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:22.433447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-41053103-e5c0-4f72-8af9-8d16e87ffa0c None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:22.434993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-41053103-e5c0-4f72-8af9-8d16e87ffa0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:22.519439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 50/99] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:22.521075 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:22.560724 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e764e-569b-4399-ba04-00c5fa110e89 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:22.628890 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575cec5-3671-44ed-a5b3-cd63c8606b56 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:22.629227 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575cec5-3671-44ed-a5b3-cd63c8606b56 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:22.629461 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b575cec5-3671-44ed-a5b3-cd63c8606b56 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:22.632083 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 51/100] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:22.638383 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1273e-22a3-433d-b28a-541283265745 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:22.638681 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1273e-22a3-433d-b28a-541283265745 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:22.638967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1273e-22a3-433d-b28a-541283265745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:22.664403 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8cc1273e-22a3-433d-b28a-541283265745 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:22.665603 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8cc1273e-22a3-433d-b28a-541283265745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:22.741948 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3e764e-569b-4399-ba04-00c5fa110e89 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:22.781877 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 52/101] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 10:13:22.809467 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d516c-a860-4817-bc48-33bdb41beff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:22.810281 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d516c-a860-4817-bc48-33bdb41beff5 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:22.810575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d516c-a860-4817-bc48-33bdb41beff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:22.810856 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d516c-a860-4817-bc48-33bdb41beff5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:22.823351 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1273e-22a3-433d-b28a-541283265745 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:22.840356 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51d516c-a860-4817-bc48-33bdb41beff5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:22.848979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-c51d516c-a860-4817-bc48-33bdb41beff5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:22.848979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:22.848979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:22.848979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:22.848979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:22.848979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:22.848979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:22.848979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:22.848979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:22.848979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:22.848979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:22.848979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:22.848979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:22.848979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:22.848979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:22.848979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:22.849813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:22.849813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:22.849813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:22.849813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:22.849813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:22.849813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:22.849813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:22.849813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:22.849813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:22.849813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:22.849813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:22.849813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:22.849813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:22.849813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:22.849813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:22.849813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:22.850651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:22.850651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:22.850651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:22.850651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:22.850651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:22.850651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:22.850651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:22.850651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:22.850651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:22.850651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:22.850651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:22.850651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:13:22.860621 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 50/102] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:22.867245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d516c-a860-4817-bc48-33bdb41beff5 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:22.927288 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:22.987889 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1273e-22a3-433d-b28a-541283265745 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:23.042092 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:23.044044 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d516c-a860-4817-bc48-33bdb41beff5 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:23.053401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b908f-f488-4c4e-bd53-087d419cb744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:23.055817 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b908f-f488-4c4e-bd53-087d419cb744 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:23.055817 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b908f-f488-4c4e-bd53-087d419cb744 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:23.055817 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b908f-f488-4c4e-bd53-087d419cb744 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:23.076378 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b908f-f488-4c4e-bd53-087d419cb744 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:23.093280 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-8e0b908f-f488-4c4e-bd53-087d419cb744 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:13:23.093280 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:23.093280 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:23.093280 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:23.093280 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:23.093280 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:23.093280 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:23.093280 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:23.093280 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:23.093280 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:23.093280 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:23.093280 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:23.093280 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:23.093280 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:23.093280 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:23.093280 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:23.094182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:23.094182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:23.094182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:23.094182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:23.094182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:23.094182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:23.094182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:23.094182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:23.094182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:23.094182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:23.094182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:23.094182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:23.094182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:23.094182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:23.094182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:23.094182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:23.094942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:23.094942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:23.094942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:23.094942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:23.094942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:23.094942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:23.094942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:23.094942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:23.094942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:23.094942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:23.094942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:13:23.094942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:13:23.094942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 53/103] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:13:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:23.101237 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b908f-f488-4c4e-bd53-087d419cb744 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:23.128584 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b659c2-7e45-455a-b9b3-932054ab9fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:23.129284 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b659c2-7e45-455a-b9b3-932054ab9fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:23.130094 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b659c2-7e45-455a-b9b3-932054ab9fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:23.130094 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b659c2-7e45-455a-b9b3-932054ab9fb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:23.130723 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b659c2-7e45-455a-b9b3-932054ab9fb4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:23.131038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b659c2-7e45-455a-b9b3-932054ab9fb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:23.132471 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b659c2-7e45-455a-b9b3-932054ab9fb4 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:23.169602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 51/104] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:13:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:23.180533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b659c2-7e45-455a-b9b3-932054ab9fb4 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:23.282190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b908f-f488-4c4e-bd53-087d419cb744 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:23.307044 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:23.324556 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b659c2-7e45-455a-b9b3-932054ab9fb4 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:23.326555 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750b530-2379-408e-9038-620d195c3081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:23.327248 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750b530-2379-408e-9038-620d195c3081 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:23.327618 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750b530-2379-408e-9038-620d195c3081 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:23.327909 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750b530-2379-408e-9038-620d195c3081 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:23.328741 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4750b530-2379-408e-9038-620d195c3081 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:23.329067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4750b530-2379-408e-9038-620d195c3081 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:23.330497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4750b530-2379-408e-9038-620d195c3081 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:23.352418 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 54/105] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:23.357902 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750b530-2379-408e-9038-620d195c3081 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:23.381235 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94f940-11b8-45a3-8aed-0f1d014ae4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:23.382145 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94f940-11b8-45a3-8aed-0f1d014ae4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:23.382432 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94f940-11b8-45a3-8aed-0f1d014ae4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:23.382714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec94f940-11b8-45a3-8aed-0f1d014ae4f8 None admin] PATH_INFO: `/v3/users/db5cbfa9be364e0d9c75eac6090e7967` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:23.383991 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec94f940-11b8-45a3-8aed-0f1d014ae4f8 None admin] RBAC: Authorizing `identity:get_user(user_id=db5cbfa9be364e0d9c75eac6090e7967)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:23.385484 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec94f940-11b8-45a3-8aed-0f1d014ae4f8 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:23.387165 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 52/106] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:23 2022] GET /identity/v3/users/db5cbfa9be364e0d9c75eac6090e7967 => generated 275 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:23.397213 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94f940-11b8-45a3-8aed-0f1d014ae4f8 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:23.438934 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:23.529985 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750b530-2379-408e-9038-620d195c3081 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:23.591992 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856aeff7-58b7-4169-82ea-e633088d5baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:23.593266 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856aeff7-58b7-4169-82ea-e633088d5baf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:23.593752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856aeff7-58b7-4169-82ea-e633088d5baf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:23.594224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856aeff7-58b7-4169-82ea-e633088d5baf None admin] PATH_INFO: `/v3/users/67e761d82cde42509f040f82a44c86c4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:23.595831 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856aeff7-58b7-4169-82ea-e633088d5baf None admin] RBAC: Authorizing `identity:get_user(user_id=67e761d82cde42509f040f82a44c86c4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:23.597527 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856aeff7-58b7-4169-82ea-e633088d5baf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:23.600120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 55/107] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:23 2022] GET /identity/v3/users/67e761d82cde42509f040f82a44c86c4 => generated 247 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:23.605963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856aeff7-58b7-4169-82ea-e633088d5baf None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:23.629230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec94f940-11b8-45a3-8aed-0f1d014ae4f8 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:23.685094 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2feab6d-4af8-4a88-a23c-95497a79600b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:23.685982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2feab6d-4af8-4a88-a23c-95497a79600b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:23.686274 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2feab6d-4af8-4a88-a23c-95497a79600b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:23.686560 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2feab6d-4af8-4a88-a23c-95497a79600b None admin] PATH_INFO: `/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:23.687815 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2feab6d-4af8-4a88-a23c-95497a79600b None admin] RBAC: Authorizing `identity:get_project(project_id=47885b7cd44548ce92d8cfb4b55c8a97)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:23.689358 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2feab6d-4af8-4a88-a23c-95497a79600b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:23.691048 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 53/108] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:23 2022] GET /identity/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97 => generated 298 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:23.697808 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2feab6d-4af8-4a88-a23c-95497a79600b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:23.747639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:23.822672 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856aeff7-58b7-4169-82ea-e633088d5baf None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:23.844152 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:23.874043 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2feab6d-4af8-4a88-a23c-95497a79600b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:23.892926 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f4071-d3fe-4857-92db-49165705325e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:23.893809 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f4071-d3fe-4857-92db-49165705325e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:23.894096 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f4071-d3fe-4857-92db-49165705325e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:23.894377 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f4071-d3fe-4857-92db-49165705325e None admin] PATH_INFO: `/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:23.900058 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f4071-d3fe-4857-92db-49165705325e None admin] RBAC: Authorizing `identity:get_project(project_id=47885b7cd44548ce92d8cfb4b55c8a97)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:23.901554 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8f4071-d3fe-4857-92db-49165705325e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:23.903368 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 56/109] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:23 2022] GET /identity/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97 => generated 298 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:23.909823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f4071-d3fe-4857-92db-49165705325e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:23.948521 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f168a993-6261-468f-a455-9a717fd25eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:23.949891 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168a993-6261-468f-a455-9a717fd25eef None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:23.950436 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168a993-6261-468f-a455-9a717fd25eef None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:23.950964 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f168a993-6261-468f-a455-9a717fd25eef None admin] PATH_INFO: `/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97/users/db5cbfa9be364e0d9c75eac6090e7967/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:23.955693 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168a993-6261-468f-a455-9a717fd25eef None admin] RBAC: Authorizing `identity:create_grant(project_id=47885b7cd44548ce92d8cfb4b55c8a97, user_id=db5cbfa9be364e0d9c75eac6090e7967, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:23.957462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f168a993-6261-468f-a455-9a717fd25eef None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:24.025712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f4071-d3fe-4857-92db-49165705325e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:24.052712 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 54/110] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jun 30 10:13:23 2022] PUT /identity/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97/users/db5cbfa9be364e0d9c75eac6090e7967/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:13:24.060026 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1afe4-8c77-4b9e-91f3-cf264cba11c7 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:24.060026 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1afe4-8c77-4b9e-91f3-cf264cba11c7 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:24.060026 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1afe4-8c77-4b9e-91f3-cf264cba11c7 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:24.060026 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 55/111] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:24.062153 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca70a59-60bf-49a1-a80d-5ae32573a5d3 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:24.062448 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca70a59-60bf-49a1-a80d-5ae32573a5d3 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:24.062729 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca70a59-60bf-49a1-a80d-5ae32573a5d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:24.084053 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-eca70a59-60bf-49a1-a80d-5ae32573a5d3 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:24.085235 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-eca70a59-60bf-49a1-a80d-5ae32573a5d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:24.090836 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293369a9-14f3-404e-8dd8-53ae68ef057c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:24.091656 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293369a9-14f3-404e-8dd8-53ae68ef057c None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:24.091948 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293369a9-14f3-404e-8dd8-53ae68ef057c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:24.092229 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293369a9-14f3-404e-8dd8-53ae68ef057c None admin] PATH_INFO: `/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97/users/67e761d82cde42509f040f82a44c86c4/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:24.094299 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293369a9-14f3-404e-8dd8-53ae68ef057c None admin] RBAC: Authorizing `identity:create_grant(project_id=47885b7cd44548ce92d8cfb4b55c8a97, user_id=67e761d82cde42509f040f82a44c86c4, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:24.095848 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293369a9-14f3-404e-8dd8-53ae68ef057c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:24.170939 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 56/112] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:24.172524 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470a5a6-6951-4e71-a8fb-eb6bac7b8f30 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:24.172819 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470a5a6-6951-4e71-a8fb-eb6bac7b8f30 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:24.173102 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470a5a6-6951-4e71-a8fb-eb6bac7b8f30 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:24.174435 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 57/113] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:24.175660 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb451ebb-e3e3-4410-afd5-c1f1249c9bfe None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:24.176012 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb451ebb-e3e3-4410-afd5-c1f1249c9bfe None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:24.176300 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb451ebb-e3e3-4410-afd5-c1f1249c9bfe None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:24.177860 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 58/114] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:24.187749 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813576a6-5b23-4362-8cba-ad7cdf6e2cbd None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:24.191735 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813576a6-5b23-4362-8cba-ad7cdf6e2cbd None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:24.191735 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813576a6-5b23-4362-8cba-ad7cdf6e2cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:24.194673 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 57/115] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jun 30 10:13:23 2022] PUT /identity/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97/users/67e761d82cde42509f040f82a44c86c4/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:13:24.196337 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79036b-753d-42ad-9209-fee92abea854 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:24.196737 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79036b-753d-42ad-9209-fee92abea854 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:24.197121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79036b-753d-42ad-9209-fee92abea854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:24.226160 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9e79036b-753d-42ad-9209-fee92abea854 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:24.227149 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9e79036b-753d-42ad-9209-fee92abea854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:24.229985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-813576a6-5b23-4362-8cba-ad7cdf6e2cbd None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:24.231295 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-813576a6-5b23-4362-8cba-ad7cdf6e2cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:24.262695 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:24.267555 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:24.311624 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 58/116] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:24.312499 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79036b-753d-42ad-9209-fee92abea854 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:24.343383 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 59/117] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:24.366434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813576a6-5b23-4362-8cba-ad7cdf6e2cbd None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:24.435826 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79036b-753d-42ad-9209-fee92abea854 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:24.493522 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c029502-9997-4793-82bc-744bed123cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:24.495109 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c029502-9997-4793-82bc-744bed123cae None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:24.495625 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c029502-9997-4793-82bc-744bed123cae None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:24.496217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c029502-9997-4793-82bc-744bed123cae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:24.497738 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c029502-9997-4793-82bc-744bed123cae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:24.504676 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c029502-9997-4793-82bc-744bed123cae None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:24.505725 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813576a6-5b23-4362-8cba-ad7cdf6e2cbd None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:24.506957 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 59/118] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:13:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:24.508319 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c029502-9997-4793-82bc-744bed123cae None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:24.570145 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:24.579228 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2fcf3-66e0-4290-a713-da226b4ee226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:24.580375 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2fcf3-66e0-4290-a713-da226b4ee226 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:24.580899 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2fcf3-66e0-4290-a713-da226b4ee226 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:24.581387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2fcf3-66e0-4290-a713-da226b4ee226 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:24.606808 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2fcf3-66e0-4290-a713-da226b4ee226 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:24.616567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-04c2fcf3-66e0-4290-a713-da226b4ee226 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 10:13:24.616567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:24.616567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:24.616567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:24.616567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:24.616567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:24.616567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:24.616567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:24.616567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:24.616567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:24.616567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:24.616567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:24.616567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:24.616567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:24.616567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:24.616567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:24.617596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:24.617596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:24.617596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:24.617596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:24.617596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:24.617596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:24.617596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:24.617596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:24.617596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:24.617596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:24.617596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:24.617596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:24.617596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:24.617596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:24.617596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:24.617596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:24.618377 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:24.618377 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:24.618377 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:24.618377 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:24.618377 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:24.618377 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:24.618377 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:24.618377 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:24.618377 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:24.618377 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:24.618377 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 10:13:24.618377 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:13:24.627309 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 60/119] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:24.629135 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2fcf3-66e0-4290-a713-da226b4ee226 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:24.650162 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:24.675480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c029502-9997-4793-82bc-744bed123cae None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:24.715662 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd06f6-372a-4ada-80c4-02db034afc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:24.716954 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd06f6-372a-4ada-80c4-02db034afc26 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:24.717488 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd06f6-372a-4ada-80c4-02db034afc26 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:24.717951 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd06f6-372a-4ada-80c4-02db034afc26 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:24.739305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd06f6-372a-4ada-80c4-02db034afc26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:24.739305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-dfbd06f6-372a-4ada-80c4-02db034afc26 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:24.739305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:24.739305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:24.739305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:24.739305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:24.739305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:24.739305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:24.739305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:24.739305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:24.739305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:24.739305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:24.739305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:24.739305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:24.740295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:24.740295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:24.740295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:24.740295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:24.740295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:24.740295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:24.740295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:24.740295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:24.740295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:24.740295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:24.740295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:24.740295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:24.740295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:24.740295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:24.740295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:24.740295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:24.740295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:24.741415 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:24.741415 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:24.741415 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:24.741415 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:24.741415 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:24.741415 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:24.741415 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:24.741415 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:24.741415 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:24.741415 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:24.741415 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:24.741415 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:24.741415 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:24.741415 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:13:24.741415 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 60/120] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:24.742325 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd06f6-372a-4ada-80c4-02db034afc26 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:24.758775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2fcf3-66e0-4290-a713-da226b4ee226 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:24.806222 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b4668d-7bcc-4e3e-bfaf-f74104215287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:24.806950 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4668d-7bcc-4e3e-bfaf-f74104215287 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:24.807243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4668d-7bcc-4e3e-bfaf-f74104215287 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:24.807550 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b4668d-7bcc-4e3e-bfaf-f74104215287 None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:24.808561 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b4668d-7bcc-4e3e-bfaf-f74104215287 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:24.810366 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b4668d-7bcc-4e3e-bfaf-f74104215287 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:24.863505 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:24.870385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd06f6-372a-4ada-80c4-02db034afc26 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:24.903594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff7183a-3af9-4643-b974-3e2fcbd2613d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:24.904272 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7183a-3af9-4643-b974-3e2fcbd2613d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:24.904561 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7183a-3af9-4643-b974-3e2fcbd2613d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:24.904844 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff7183a-3af9-4643-b974-3e2fcbd2613d None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:24.905605 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7183a-3af9-4643-b974-3e2fcbd2613d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:24.905918 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7183a-3af9-4643-b974-3e2fcbd2613d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:24.907568 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff7183a-3af9-4643-b974-3e2fcbd2613d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:24.919344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 61/121] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:13:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:24.919344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ac703-eb85-424f-ac69-0d169d7c48fe None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:24.919344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ac703-eb85-424f-ac69-0d169d7c48fe None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:24.919344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ac703-eb85-424f-ac69-0d169d7c48fe None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:24.920343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 61/122] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jun 30 10:13:24 2022] POST /identity/v3/projects => generated 302 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:13:24.920629 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 62/123] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:24.931498 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ac703-eb85-424f-ac69-0d169d7c48fe None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:24.932907 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b4668d-7bcc-4e3e-bfaf-f74104215287 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:25.011718 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:25.062738 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b4668d-7bcc-4e3e-bfaf-f74104215287 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:25.106621 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b130aa-f45b-4138-9a02-1aa444fd6c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:25.107989 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b130aa-f45b-4138-9a02-1aa444fd6c6f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:25.108523 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b130aa-f45b-4138-9a02-1aa444fd6c6f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:25.109553 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b130aa-f45b-4138-9a02-1aa444fd6c6f None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:25.110859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b130aa-f45b-4138-9a02-1aa444fd6c6f None admin] RBAC: Adding query filter params (name=member) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:25.111640 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b130aa-f45b-4138-9a02-1aa444fd6c6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:25.114207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b130aa-f45b-4138-9a02-1aa444fd6c6f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:25.125810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 62/124] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:13:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:25.128123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45159a53-973c-4ca0-8e64-32c0eda494e3 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:25.132653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45159a53-973c-4ca0-8e64-32c0eda494e3 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:25.132653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45159a53-973c-4ca0-8e64-32c0eda494e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:25.166850 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-45159a53-973c-4ca0-8e64-32c0eda494e3 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:25.168648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-45159a53-973c-4ca0-8e64-32c0eda494e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:25.178838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9ac703-eb85-424f-ac69-0d169d7c48fe None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:25.245749 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d78169a-33b7-4253-a781-c26340c4e62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:25.246262 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78169a-33b7-4253-a781-c26340c4e62f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:25.246561 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78169a-33b7-4253-a781-c26340c4e62f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:25.246848 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d78169a-33b7-4253-a781-c26340c4e62f None admin] PATH_INFO: `/v3/users/2377ca134c2c480fbe482fc72aa2b0d6` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:25.263862 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d78169a-33b7-4253-a781-c26340c4e62f None admin] RBAC: Authorizing `identity:get_user(user_id=2377ca134c2c480fbe482fc72aa2b0d6)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:25.266558 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d78169a-33b7-4253-a781-c26340c4e62f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:25.267901 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 63/125] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:24 2022] GET /identity/v3/users/2377ca134c2c480fbe482fc72aa2b0d6 => generated 289 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:25.268772 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d78169a-33b7-4253-a781-c26340c4e62f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:25.288431 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 63/126] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:25.296070 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45159a53-973c-4ca0-8e64-32c0eda494e3 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:25.296560 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:25.371698 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:25.390607 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d78169a-33b7-4253-a781-c26340c4e62f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:25.433266 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45159a53-973c-4ca0-8e64-32c0eda494e3 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:25.465874 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf19674-c5df-42bd-a0c1-4fdddffb7779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:25.466766 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf19674-c5df-42bd-a0c1-4fdddffb7779 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:25.467059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf19674-c5df-42bd-a0c1-4fdddffb7779 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:25.467372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf19674-c5df-42bd-a0c1-4fdddffb7779 None admin] PATH_INFO: `/v3/users/db5cbfa9be364e0d9c75eac6090e7967` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:25.469977 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf19674-c5df-42bd-a0c1-4fdddffb7779 None admin] RBAC: Authorizing `identity:get_user(user_id=db5cbfa9be364e0d9c75eac6090e7967)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:25.471484 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf19674-c5df-42bd-a0c1-4fdddffb7779 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:25.473802 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 64/127] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:25 2022] GET /identity/v3/users/db5cbfa9be364e0d9c75eac6090e7967 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:25.479051 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf19674-c5df-42bd-a0c1-4fdddffb7779 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:25.529614 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3b891-8e09-4922-9781-bcc25c9dc43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:25.530209 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3b891-8e09-4922-9781-bcc25c9dc43c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:25.530496 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3b891-8e09-4922-9781-bcc25c9dc43c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:25.530778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f3b891-8e09-4922-9781-bcc25c9dc43c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:25.639733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf19674-c5df-42bd-a0c1-4fdddffb7779 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:25.662154 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f3b891-8e09-4922-9781-bcc25c9dc43c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:25.667709 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:25.667845 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f3b891-8e09-4922-9781-bcc25c9dc43c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:25.681270 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247769a2-8d01-495f-8821-97b35e143e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:25.682257 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247769a2-8d01-495f-8821-97b35e143e9c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:25.682548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247769a2-8d01-495f-8821-97b35e143e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:25.682829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247769a2-8d01-495f-8821-97b35e143e9c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:25.705198 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247769a2-8d01-495f-8821-97b35e143e9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:25.715091 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-247769a2-8d01-495f-8821-97b35e143e9c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 10:13:25.715091 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:25.715091 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:25.715091 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:25.715091 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:25.715091 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:25.715091 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:25.715091 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:25.715091 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:25.715091 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:25.715091 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:25.715091 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:25.715091 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:25.715091 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:25.715091 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:25.715091 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:25.715982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:25.715982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:25.715982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:25.715982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:25.715982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:25.715982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:25.715982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:25.715982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:25.715982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:25.715982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:25.715982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:25.715982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:25.715982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:25.715982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:25.715982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:25.715982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:25.716740 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:25.716740 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:25.716740 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:25.716740 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:25.716740 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:25.716740 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:25.716740 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:25.716740 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:25.716740 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:25.716740 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:25.716740 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 10:13:25.716740 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:13:25.721075 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 65/128] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jun 30 10:13:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:25.722663 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247769a2-8d01-495f-8821-97b35e143e9c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:25.775605 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:25.842836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 64/129] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:13:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:13:25.844284 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3b891-8e09-4922-9781-bcc25c9dc43c None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:25.922189 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247769a2-8d01-495f-8821-97b35e143e9c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:25.990212 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f3b891-8e09-4922-9781-bcc25c9dc43c None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:25.994373 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b1b6b-23fa-4f1f-b049-f60da1190c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:25.995251 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b1b6b-23fa-4f1f-b049-f60da1190c1c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:25.995569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b1b6b-23fa-4f1f-b049-f60da1190c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:25.995855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33b1b6b-23fa-4f1f-b049-f60da1190c1c None admin] PATH_INFO: `/v3/projects/ce606c0a9d6547819fdbd91ffce253f6` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:26.002096 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b1b6b-23fa-4f1f-b049-f60da1190c1c None admin] RBAC: Authorizing `identity:get_project(project_id=ce606c0a9d6547819fdbd91ffce253f6)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:26.003633 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33b1b6b-23fa-4f1f-b049-f60da1190c1c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:26.006316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 66/130] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:25 2022] GET /identity/v3/projects/ce606c0a9d6547819fdbd91ffce253f6 => generated 312 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:26.007112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b1b6b-23fa-4f1f-b049-f60da1190c1c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:26.034507 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:26.050089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfc6b7-81c4-410d-a719-b0cbbf067b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:26.050780 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfc6b7-81c4-410d-a719-b0cbbf067b7d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:26.051073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfc6b7-81c4-410d-a719-b0cbbf067b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:26.051496 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfc6b7-81c4-410d-a719-b0cbbf067b7d None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:26.052265 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debfc6b7-81c4-410d-a719-b0cbbf067b7d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:26.052582 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debfc6b7-81c4-410d-a719-b0cbbf067b7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:26.053981 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debfc6b7-81c4-410d-a719-b0cbbf067b7d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:26.072170 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 65/131] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:13:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:26.072785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfc6b7-81c4-410d-a719-b0cbbf067b7d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:26.116201 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:26.173440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33b1b6b-23fa-4f1f-b049-f60da1190c1c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:26.221853 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfc6b7-81c4-410d-a719-b0cbbf067b7d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:26.262359 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d781495-8906-40fd-902b-80632b8a9cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:26.263284 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d781495-8906-40fd-902b-80632b8a9cfc None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:26.263576 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d781495-8906-40fd-902b-80632b8a9cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:26.263862 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d781495-8906-40fd-902b-80632b8a9cfc None admin] PATH_INFO: `/v3/projects/ce606c0a9d6547819fdbd91ffce253f6/users/db5cbfa9be364e0d9c75eac6090e7967/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:26.266605 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d781495-8906-40fd-902b-80632b8a9cfc None admin] RBAC: Authorizing `identity:create_grant(project_id=ce606c0a9d6547819fdbd91ffce253f6, user_id=db5cbfa9be364e0d9c75eac6090e7967, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:26.269064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d781495-8906-40fd-902b-80632b8a9cfc None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:26.279956 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d83d67-e449-449e-b25f-edc21edbb12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:26.280836 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d83d67-e449-449e-b25f-edc21edbb12e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:26.281121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d83d67-e449-449e-b25f-edc21edbb12e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:26.281443 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d83d67-e449-449e-b25f-edc21edbb12e None admin] PATH_INFO: `/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:26.282642 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d83d67-e449-449e-b25f-edc21edbb12e None admin] RBAC: Authorizing `identity:get_project(project_id=47885b7cd44548ce92d8cfb4b55c8a97)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:26.284108 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d83d67-e449-449e-b25f-edc21edbb12e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:26.287960 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 67/132] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:26 2022] GET /identity/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97 => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:26.287960 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d83d67-e449-449e-b25f-edc21edbb12e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:26.340934 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 66/133] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jun 30 10:13:26 2022] PUT /identity/v3/projects/ce606c0a9d6547819fdbd91ffce253f6/users/db5cbfa9be364e0d9c75eac6090e7967/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:13:26.342458 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d781495-8906-40fd-902b-80632b8a9cfc None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:26.443660 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:26.456492 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:26.484142 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d781495-8906-40fd-902b-80632b8a9cfc None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:26.506733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d83d67-e449-449e-b25f-edc21edbb12e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:26.569302 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98cab4-10c5-4352-a229-fa7b90cc9b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:26.570235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98cab4-10c5-4352-a229-fa7b90cc9b87 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:26.570526 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98cab4-10c5-4352-a229-fa7b90cc9b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:26.570809 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98cab4-10c5-4352-a229-fa7b90cc9b87 None admin] PATH_INFO: `/v3/users/db5cbfa9be364e0d9c75eac6090e7967` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:26.572796 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a4f60a-2ae1-4c0b-a005-ef857464bdbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:26.573508 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4f60a-2ae1-4c0b-a005-ef857464bdbf None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:26.573804 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4f60a-2ae1-4c0b-a005-ef857464bdbf None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:26.574092 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a4f60a-2ae1-4c0b-a005-ef857464bdbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:26.574982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4f60a-2ae1-4c0b-a005-ef857464bdbf None admin] RBAC: Adding query filter params (role.id=ea0b118209d448f2902b045a6ac12919, scope.project.id=47885b7cd44548ce92d8cfb4b55c8a97, user.id=2377ca134c2c480fbe482fc72aa2b0d6) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:26.575332 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4f60a-2ae1-4c0b-a005-ef857464bdbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:26.576217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98cab4-10c5-4352-a229-fa7b90cc9b87 None admin] RBAC: Authorizing `identity:get_user(user_id=db5cbfa9be364e0d9c75eac6090e7967)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:26.578045 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98cab4-10c5-4352-a229-fa7b90cc9b87 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:26.579996 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 68/134] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:26 2022] GET /identity/v3/users/db5cbfa9be364e0d9c75eac6090e7967 => generated 275 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:26.584201 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a4f60a-2ae1-4c0b-a005-ef857464bdbf None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:26.585804 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98cab4-10c5-4352-a229-fa7b90cc9b87 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:26.593566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 67/135] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:13:26 2022] GET /identity/v3/role_assignments?user.id=2377ca134c2c480fbe482fc72aa2b0d6&scope.project.id=47885b7cd44548ce92d8cfb4b55c8a97&role.id=ea0b118209d448f2902b045a6ac12919 => generated 259 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:26.627605 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc95dc-65ac-4e6c-99cb-7c2e1c03e5f6 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:26.627919 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc95dc-65ac-4e6c-99cb-7c2e1c03e5f6 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:26.628192 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc95dc-65ac-4e6c-99cb-7c2e1c03e5f6 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:26.630137 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 68/136] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:26.640852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c30214-7111-46de-95c7-7bb1b468f9d2 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:26.641167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c30214-7111-46de-95c7-7bb1b468f9d2 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:26.641504 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c30214-7111-46de-95c7-7bb1b468f9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:26.669793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-e4c30214-7111-46de-95c7-7bb1b468f9d2 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:26.671032 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-e4c30214-7111-46de-95c7-7bb1b468f9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:26.704889 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98cab4-10c5-4352-a229-fa7b90cc9b87 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:26.742653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528e81bb-b498-410e-863e-3ad9463bdaa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:26.743586 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e81bb-b498-410e-863e-3ad9463bdaa0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:26.743873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e81bb-b498-410e-863e-3ad9463bdaa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:26.744154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528e81bb-b498-410e-863e-3ad9463bdaa0 None admin] PATH_INFO: `/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:26.749841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528e81bb-b498-410e-863e-3ad9463bdaa0 None admin] RBAC: Authorizing `identity:get_project(project_id=47885b7cd44548ce92d8cfb4b55c8a97)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:26.754628 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528e81bb-b498-410e-863e-3ad9463bdaa0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:26.758574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 69/137] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:26 2022] GET /identity/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:26.760005 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407e1e4-6b88-4657-8a01-0059c28387ee None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:26.760536 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407e1e4-6b88-4657-8a01-0059c28387ee None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:26.760819 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4407e1e4-6b88-4657-8a01-0059c28387ee None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:26.762158 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 70/138] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:26.762854 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4407e1e4-6b88-4657-8a01-0059c28387ee None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:26.784154 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 69/139] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:26.785502 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11ed26-e630-4a21-98a3-93c40ab48fc2 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:26.786356 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11ed26-e630-4a21-98a3-93c40ab48fc2 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:26.786756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11ed26-e630-4a21-98a3-93c40ab48fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:26.804949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:26.812589 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-ce11ed26-e630-4a21-98a3-93c40ab48fc2 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:26.814892 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-ce11ed26-e630-4a21-98a3-93c40ab48fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:26.908841 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:26.919289 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 70/140] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:26.919289 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce11ed26-e630-4a21-98a3-93c40ab48fc2 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:26.939986 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4407e1e4-6b88-4657-8a01-0059c28387ee None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:26.985460 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13112507-683a-4e86-9621-3eead40d24da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:26.986760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13112507-683a-4e86-9621-3eead40d24da None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:26.987361 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13112507-683a-4e86-9621-3eead40d24da None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:26.987854 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13112507-683a-4e86-9621-3eead40d24da None admin] PATH_INFO: `/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97/users/db5cbfa9be364e0d9c75eac6090e7967/roles/284d1573e06d47eba0196afbe92d4d7d` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:26.995018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13112507-683a-4e86-9621-3eead40d24da None admin] RBAC: Authorizing `identity:create_grant(project_id=47885b7cd44548ce92d8cfb4b55c8a97, user_id=db5cbfa9be364e0d9c75eac6090e7967, role_id=284d1573e06d47eba0196afbe92d4d7d)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:27.000162 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13112507-683a-4e86-9621-3eead40d24da None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:27.031241 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce11ed26-e630-4a21-98a3-93c40ab48fc2 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:27.073285 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570914b-df81-4d2a-9a27-c583329606d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:27.076295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 71/141] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jun 30 10:13:26 2022] PUT /identity/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97/users/db5cbfa9be364e0d9c75eac6090e7967/roles/284d1573e06d47eba0196afbe92d4d7d => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:13:27.077521 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13112507-683a-4e86-9621-3eead40d24da None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:27.081600 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570914b-df81-4d2a-9a27-c583329606d7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:27.083391 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570914b-df81-4d2a-9a27-c583329606d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:27.083885 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570914b-df81-4d2a-9a27-c583329606d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:27.105616 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4570914b-df81-4d2a-9a27-c583329606d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:27.118009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-4570914b-df81-4d2a-9a27-c583329606d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:13:27.118009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:27.118009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:27.118009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:27.118009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:27.118009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:27.118009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:27.118009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:27.118009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:27.118009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:27.118009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:27.118009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:27.118009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:27.118009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:27.118009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:27.118009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:27.119035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:27.119035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:27.119035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:27.119035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:27.119035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:27.119035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:27.119035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:27.119035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:27.119035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:27.119035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:27.119035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:27.119035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:27.119035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:27.119035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:27.119035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:27.119035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:27.120314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:27.120314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:27.120314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:27.120314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:27.120314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:27.120314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:27.120314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:27.120314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:27.120314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:27.120314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:27.120314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:13:27.120314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:13:27.120314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 71/142] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:13:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:27.133208 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570914b-df81-4d2a-9a27-c583329606d7 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:27.239818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:27.269513 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13112507-683a-4e86-9621-3eead40d24da None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:27.321754 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:27.326880 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5154f-dd4e-4673-88ea-3974f3be2584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:27.327866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5154f-dd4e-4673-88ea-3974f3be2584 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:27.328268 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5154f-dd4e-4673-88ea-3974f3be2584 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:27.333096 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c5154f-dd4e-4673-88ea-3974f3be2584 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:27.342090 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c5154f-dd4e-4673-88ea-3974f3be2584 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:27.363380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-39c5154f-dd4e-4673-88ea-3974f3be2584 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:27.363380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:27.363380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:27.363380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:27.363380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:27.363380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:27.363380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:27.363380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:27.363380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:27.363380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:27.363380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:27.363380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:27.363380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:27.363380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:27.363380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:27.363380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:27.364408 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:27.364408 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:27.364408 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:27.364408 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:27.364408 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:27.364408 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:27.364408 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:27.364408 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:27.364408 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:27.364408 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:27.364408 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:27.364408 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:27.364408 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:27.364408 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:27.364408 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:27.364408 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:27.365693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:27.365693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:27.365693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:27.365693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:27.365693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:27.365693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:27.365693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:27.365693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:27.365693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:27.365693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:27.365693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:27.365693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:13:27.365693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 72/143] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:27.372424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5154f-dd4e-4673-88ea-3974f3be2584 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:27.409174 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570914b-df81-4d2a-9a27-c583329606d7 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:27.463615 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c39797-9f08-49dc-9370-a986d9326b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:27.464318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c39797-9f08-49dc-9370-a986d9326b30 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:27.464610 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c39797-9f08-49dc-9370-a986d9326b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:27.464890 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c39797-9f08-49dc-9370-a986d9326b30 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:27.465665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c39797-9f08-49dc-9370-a986d9326b30 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:27.465982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c39797-9f08-49dc-9370-a986d9326b30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:27.467410 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c39797-9f08-49dc-9370-a986d9326b30 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:27.495195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 72/144] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 362 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:27.507473 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c39797-9f08-49dc-9370-a986d9326b30 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:27.555944 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c5154f-dd4e-4673-88ea-3974f3be2584 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:27.633726 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b55ef-e3e2-4007-acdd-09fe5b2a0502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:27.634407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b55ef-e3e2-4007-acdd-09fe5b2a0502 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:27.634690 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b55ef-e3e2-4007-acdd-09fe5b2a0502 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:27.634965 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b55ef-e3e2-4007-acdd-09fe5b2a0502 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:27.635755 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b55ef-e3e2-4007-acdd-09fe5b2a0502 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:27.636061 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b55ef-e3e2-4007-acdd-09fe5b2a0502 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:27.637768 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b55ef-e3e2-4007-acdd-09fe5b2a0502 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:27.651617 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:27.657419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 73/145] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:13:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:27.663930 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b55ef-e3e2-4007-acdd-09fe5b2a0502 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:27.747110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c39797-9f08-49dc-9370-a986d9326b30 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:27.791914 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:27.827707 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f0157-82db-4e11-9129-3415e9ac0e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:27.828618 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f0157-82db-4e11-9129-3415e9ac0e77 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:27.828906 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f0157-82db-4e11-9129-3415e9ac0e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:27.829185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f0157-82db-4e11-9129-3415e9ac0e77 None admin] PATH_INFO: `/v3/users/67e761d82cde42509f040f82a44c86c4` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:27.833853 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f0157-82db-4e11-9129-3415e9ac0e77 None admin] RBAC: Authorizing `identity:get_user(user_id=67e761d82cde42509f040f82a44c86c4)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:27.835372 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f0157-82db-4e11-9129-3415e9ac0e77 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:27.837399 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 73/146] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:27 2022] GET /identity/v3/users/67e761d82cde42509f040f82a44c86c4 => generated 247 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:27.843363 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f0157-82db-4e11-9129-3415e9ac0e77 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:27.874920 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b55ef-e3e2-4007-acdd-09fe5b2a0502 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:27.928589 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb1fa12-1dde-418c-bec2-667d88637627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:27.929568 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1fa12-1dde-418c-bec2-667d88637627 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:27.929856 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1fa12-1dde-418c-bec2-667d88637627 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:27.930130 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb1fa12-1dde-418c-bec2-667d88637627 None admin] PATH_INFO: `/v3/users/db5cbfa9be364e0d9c75eac6090e7967` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:27.931777 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1fa12-1dde-418c-bec2-667d88637627 None admin] RBAC: Authorizing `identity:get_user(user_id=db5cbfa9be364e0d9c75eac6090e7967)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:27.933231 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb1fa12-1dde-418c-bec2-667d88637627 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:27.936135 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 74/147] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:27 2022] GET /identity/v3/users/db5cbfa9be364e0d9c75eac6090e7967 => generated 275 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:27.937360 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d8e96-6edb-4a33-8965-7dae35b14428 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:27.937610 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d8e96-6edb-4a33-8965-7dae35b14428 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:27.937887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963d8e96-6edb-4a33-8965-7dae35b14428 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:27.949171 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 75/148] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:27 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:27.950264 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d8e96-6edb-4a33-8965-7dae35b14428 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:28.066737 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f0157-82db-4e11-9129-3415e9ac0e77 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:28.091603 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:28.136723 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2507f0-6670-45a5-8ff5-bebccee740b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:28.137754 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2507f0-6670-45a5-8ff5-bebccee740b9 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:28.138241 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2507f0-6670-45a5-8ff5-bebccee740b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:28.138720 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2507f0-6670-45a5-8ff5-bebccee740b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:28.164309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963d8e96-6edb-4a33-8965-7dae35b14428 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:28.211911 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f5f5b-c4cf-4eca-831b-af567b1dcdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:28.213041 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f5f5b-c4cf-4eca-831b-af567b1dcdda None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:28.213527 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f5f5b-c4cf-4eca-831b-af567b1dcdda None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:28.214001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f5f5b-c4cf-4eca-831b-af567b1dcdda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:28.228057 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:28.247291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2507f0-6670-45a5-8ff5-bebccee740b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:28.247291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2507f0-6670-45a5-8ff5-bebccee740b9 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:28.341491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f5f5b-c4cf-4eca-831b-af567b1dcdda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:28.342910 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f5f5b-c4cf-4eca-831b-af567b1dcdda None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:28.387557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 74/149] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:13:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 10:13:28.388674 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf6c6a-f55e-4f53-a49d-4cf4183668f8 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:28.389110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf6c6a-f55e-4f53-a49d-4cf4183668f8 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:28.389278 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cf6c6a-f55e-4f53-a49d-4cf4183668f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:28.421623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-11cf6c6a-f55e-4f53-a49d-4cf4183668f8 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:28.424525 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-11cf6c6a-f55e-4f53-a49d-4cf4183668f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:28.463299 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:28.534256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 75/150] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:28.535112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf6c6a-f55e-4f53-a49d-4cf4183668f8 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:28.542310 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 76/151] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:13:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:13:28.548011 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f5f5b-c4cf-4eca-831b-af567b1dcdda None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:28.575681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:28.665808 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cf6c6a-f55e-4f53-a49d-4cf4183668f8 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:28.706840 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f6bdb8-78ff-4d05-941d-a47cc7445242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:28.708293 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6bdb8-78ff-4d05-941d-a47cc7445242 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:28.708810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6bdb8-78ff-4d05-941d-a47cc7445242 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:28.709246 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f6bdb8-78ff-4d05-941d-a47cc7445242 None admin] PATH_INFO: `/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:28.710698 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f6bdb8-78ff-4d05-941d-a47cc7445242 None admin] RBAC: Authorizing `identity:get_project(project_id=47885b7cd44548ce92d8cfb4b55c8a97)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:28.713262 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f6bdb8-78ff-4d05-941d-a47cc7445242 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:28.715246 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 76/152] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:28 2022] GET /identity/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:28.716219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f6bdb8-78ff-4d05-941d-a47cc7445242 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:28.745432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f5f5b-c4cf-4eca-831b-af567b1dcdda None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:28.821832 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa4acc8-8911-4901-8653-9539f6e11afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:28.822959 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4acc8-8911-4901-8653-9539f6e11afa None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:28.825477 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4acc8-8911-4901-8653-9539f6e11afa None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:28.825907 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa4acc8-8911-4901-8653-9539f6e11afa None admin] PATH_INFO: `/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:28.827949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa4acc8-8911-4901-8653-9539f6e11afa None admin] RBAC: Authorizing `identity:get_project(project_id=47885b7cd44548ce92d8cfb4b55c8a97)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:28.834352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa4acc8-8911-4901-8653-9539f6e11afa None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:28.836630 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 77/153] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:28 2022] GET /identity/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97 => generated 298 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:28.837562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa4acc8-8911-4901-8653-9539f6e11afa None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:28.892156 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:28.905793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f6bdb8-78ff-4d05-941d-a47cc7445242 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:28.907312 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:28.946677 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4726153-b661-4c19-9b7b-d1ccd9bd8019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:28.947919 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4726153-b661-4c19-9b7b-d1ccd9bd8019 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:28.948381 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4726153-b661-4c19-9b7b-d1ccd9bd8019 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:28.948788 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4726153-b661-4c19-9b7b-d1ccd9bd8019 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:28.950103 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4726153-b661-4c19-9b7b-d1ccd9bd8019 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:28.951830 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4726153-b661-4c19-9b7b-d1ccd9bd8019 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:28.953674 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 77/154] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:13:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:28.954562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4726153-b661-4c19-9b7b-d1ccd9bd8019 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:29.005714 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa4acc8-8911-4901-8653-9539f6e11afa None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:29.052721 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8355858-2549-41a0-a544-3075db2737d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:29.053861 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8355858-2549-41a0-a544-3075db2737d7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:29.054315 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8355858-2549-41a0-a544-3075db2737d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:29.054864 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8355858-2549-41a0-a544-3075db2737d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:29.056202 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8355858-2549-41a0-a544-3075db2737d7 None admin] RBAC: Adding query filter params (role.id=e63bda042e104a4a93fba15ffcfe1183, scope.project.id=47885b7cd44548ce92d8cfb4b55c8a97, user.id=67e761d82cde42509f040f82a44c86c4) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:29.056670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8355858-2549-41a0-a544-3075db2737d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:29.058667 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8355858-2549-41a0-a544-3075db2737d7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:29.066261 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 78/155] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:13:28 2022] GET /identity/v3/role_assignments?user.id=67e761d82cde42509f040f82a44c86c4&scope.project.id=47885b7cd44548ce92d8cfb4b55c8a97&role.id=e63bda042e104a4a93fba15ffcfe1183 => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:29.067764 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8355858-2549-41a0-a544-3075db2737d7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:29.072963 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4726153-b661-4c19-9b7b-d1ccd9bd8019 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:29.122287 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eabf093-7028-4491-b7c7-d903937c0686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:29.122967 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabf093-7028-4491-b7c7-d903937c0686 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:29.123255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabf093-7028-4491-b7c7-d903937c0686 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:29.123579 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eabf093-7028-4491-b7c7-d903937c0686 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:29.124832 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eabf093-7028-4491-b7c7-d903937c0686 None admin] RBAC: Adding query filter params (role.id=31034f95ef88402296c753c2c259f717, scope.project.id=47885b7cd44548ce92d8cfb4b55c8a97, user.id=db5cbfa9be364e0d9c75eac6090e7967) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:29.125155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eabf093-7028-4491-b7c7-d903937c0686 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:29.127135 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eabf093-7028-4491-b7c7-d903937c0686 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:29.135556 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 78/156] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:13:28 2022] GET /identity/v3/role_assignments?user.id=db5cbfa9be364e0d9c75eac6090e7967&scope.project.id=47885b7cd44548ce92d8cfb4b55c8a97&role.id=31034f95ef88402296c753c2c259f717 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:29.136961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0cd77-7548-48ff-8126-cf49a0a1475a None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:29.137254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0cd77-7548-48ff-8126-cf49a0a1475a None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:29.137534 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0cd77-7548-48ff-8126-cf49a0a1475a None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:29.143357 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 79/157] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:29.148367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc9129-7c2e-47d4-b48d-cb31e0b60fe3 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:29.148721 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc9129-7c2e-47d4-b48d-cb31e0b60fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:29.149143 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dc9129-7c2e-47d4-b48d-cb31e0b60fe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:29.192306 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-f2dc9129-7c2e-47d4-b48d-cb31e0b60fe3 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:29.193555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-f2dc9129-7c2e-47d4-b48d-cb31e0b60fe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:29.231416 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:29.254068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:29.259838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8355858-2549-41a0-a544-3075db2737d7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:29.307936 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac377d-5901-4b77-9f02-9ee85d1c9c45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:29.308687 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac377d-5901-4b77-9f02-9ee85d1c9c45 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:29.308925 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac377d-5901-4b77-9f02-9ee85d1c9c45 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:29.309205 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ac377d-5901-4b77-9f02-9ee85d1c9c45 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:29.311095 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac377d-5901-4b77-9f02-9ee85d1c9c45 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:29.312967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ac377d-5901-4b77-9f02-9ee85d1c9c45 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:29.354564 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 80/158] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:29.356364 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d1bd3-16ee-44a0-aeb7-e5509cc1cd3c None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:29.356714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d1bd3-16ee-44a0-aeb7-e5509cc1cd3c None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:29.357054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494d1bd3-16ee-44a0-aeb7-e5509cc1cd3c None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:29.358631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 81/159] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:29.364728 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fb625-530c-4737-89ef-6ca8493ac2aa None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:29.365258 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fb625-530c-4737-89ef-6ca8493ac2aa None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:29.365608 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90fb625-530c-4737-89ef-6ca8493ac2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:29.384916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-d90fb625-530c-4737-89ef-6ca8493ac2aa None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:29.386165 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d90fb625-530c-4737-89ef-6ca8493ac2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:29.422742 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 79/160] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jun 30 10:13:29 2022] POST /identity/v3/users => generated 283 bytes in 356 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:13:29.424274 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac377d-5901-4b77-9f02-9ee85d1c9c45 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:29.476476 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 82/161] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:29.516547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90fb625-530c-4737-89ef-6ca8493ac2aa None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:29.565842 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ac377d-5901-4b77-9f02-9ee85d1c9c45 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:29.617140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb6d6b-3b27-4db2-b4d5-6a7356533b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:29.618640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb6d6b-3b27-4db2-b4d5-6a7356533b88 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:29.619206 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb6d6b-3b27-4db2-b4d5-6a7356533b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:29.619525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb6d6b-3b27-4db2-b4d5-6a7356533b88 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:29.634579 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb6d6b-3b27-4db2-b4d5-6a7356533b88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:29.636383 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:29.645386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-6dfb6d6b-3b27-4db2-b4d5-6a7356533b88 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:29.645386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:29.645386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:29.645386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:29.645386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:29.645386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:29.645386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:29.645386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:29.645386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:29.645386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:29.645386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:29.645386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:29.645386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:29.645386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:29.645386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:29.645386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:29.651432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:29.651432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:29.651432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:29.651432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:29.651432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:29.651432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:29.651432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:29.651432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:29.651432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:29.651432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:29.651432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:29.651432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:29.651432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:29.651432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:29.651432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:29.651432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:29.652385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:29.652385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:29.652385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:29.652385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:29.652385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:29.652385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:29.652385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:29.652385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:29.652385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:29.652385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:29.652385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:29.652385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:13:29.652385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 80/162] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:29.655399 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb6d6b-3b27-4db2-b4d5-6a7356533b88 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:29.662875 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90fb625-530c-4737-89ef-6ca8493ac2aa None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:29.679735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:29.712733 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3e443-9d65-420a-88ab-c3681012c849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:29.714205 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3e443-9d65-420a-88ab-c3681012c849 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:29.714680 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3e443-9d65-420a-88ab-c3681012c849 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:29.716210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe3e443-9d65-420a-88ab-c3681012c849 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:29.731997 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe3e443-9d65-420a-88ab-c3681012c849 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:29.740526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-0fe3e443-9d65-420a-88ab-c3681012c849 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 10:13:29.740526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:29.740526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:29.740526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:29.740526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:29.740526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:29.740526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:29.740526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:29.740526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:29.740526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:29.740526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:29.740526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:29.740526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:29.740526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:29.740526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:29.740526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:29.741441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:29.741441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:29.741441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:29.741441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:29.741441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:29.741441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:29.741441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:29.741441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:29.741441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:29.741441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:29.741441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:29.741441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:29.741441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:29.741441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:29.741441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:29.741441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:29.742210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:29.742210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:29.742210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:29.742210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:29.742210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:29.742210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:29.742210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:29.742210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:29.742210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:29.742210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:29.742210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 10:13:29.742210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:13:29.742210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 83/163] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:13:29.747819 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3e443-9d65-420a-88ab-c3681012c849 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:29.788461 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb6d6b-3b27-4db2-b4d5-6a7356533b88 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:29.833850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42bf86-70fa-4a1a-8456-8670a579dbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:29.834540 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42bf86-70fa-4a1a-8456-8670a579dbba None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:29.834831 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42bf86-70fa-4a1a-8456-8670a579dbba None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:29.835111 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe42bf86-70fa-4a1a-8456-8670a579dbba None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:29.836118 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42bf86-70fa-4a1a-8456-8670a579dbba None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:29.836431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42bf86-70fa-4a1a-8456-8670a579dbba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:29.838569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe42bf86-70fa-4a1a-8456-8670a579dbba None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:29.851359 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 81/164] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:13:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:29.858055 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42bf86-70fa-4a1a-8456-8670a579dbba None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:29.881190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe3e443-9d65-420a-88ab-c3681012c849 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:29.925810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9309be-6ab0-4385-b65f-686a9f7faea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:29.927395 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9309be-6ab0-4385-b65f-686a9f7faea3 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:29.927754 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9309be-6ab0-4385-b65f-686a9f7faea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:29.928034 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9309be-6ab0-4385-b65f-686a9f7faea3 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:29.928794 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9309be-6ab0-4385-b65f-686a9f7faea3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:29.929098 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9309be-6ab0-4385-b65f-686a9f7faea3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:29.930492 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9309be-6ab0-4385-b65f-686a9f7faea3 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:29.936450 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:29.943106 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 84/165] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:13:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:29.943636 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22deaea4-cee0-49c9-9acc-1e182c10dd2d None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:29.943740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22deaea4-cee0-49c9-9acc-1e182c10dd2d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:29.944314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22deaea4-cee0-49c9-9acc-1e182c10dd2d None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:29.946486 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 85/166] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:29.951412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22deaea4-cee0-49c9-9acc-1e182c10dd2d None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:29.965861 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:29.994474 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe42bf86-70fa-4a1a-8456-8670a579dbba None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:30.038263 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a0e5a4-d5d1-4c37-9516-a677569d6169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:30.039455 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0e5a4-d5d1-4c37-9516-a677569d6169 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:30.039703 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0e5a4-d5d1-4c37-9516-a677569d6169 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:30.040788 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a0e5a4-d5d1-4c37-9516-a677569d6169 None admin] PATH_INFO: `/v3/users/db5cbfa9be364e0d9c75eac6090e7967` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:30.042075 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0e5a4-d5d1-4c37-9516-a677569d6169 None admin] RBAC: Authorizing `identity:get_user(user_id=db5cbfa9be364e0d9c75eac6090e7967)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:30.044119 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a0e5a4-d5d1-4c37-9516-a677569d6169 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:30.047391 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 82/167] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:29 2022] GET /identity/v3/users/db5cbfa9be364e0d9c75eac6090e7967 => generated 275 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:30.048906 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8042b7-f67b-492d-a38c-582de48e40e8 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:30.049363 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8042b7-f67b-492d-a38c-582de48e40e8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:30.049810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8042b7-f67b-492d-a38c-582de48e40e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:30.068036 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-dc8042b7-f67b-492d-a38c-582de48e40e8 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:30.069286 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-dc8042b7-f67b-492d-a38c-582de48e40e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:30.079217 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22deaea4-cee0-49c9-9acc-1e182c10dd2d None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:30.130118 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679fd8c6-eb32-499b-a122-111f32642bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:30.131088 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fd8c6-eb32-499b-a122-111f32642bde None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:30.131638 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fd8c6-eb32-499b-a122-111f32642bde None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:30.131924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679fd8c6-eb32-499b-a122-111f32642bde None admin] PATH_INFO: `/v3/users/2377ca134c2c480fbe482fc72aa2b0d6` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:30.133384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679fd8c6-eb32-499b-a122-111f32642bde None admin] RBAC: Authorizing `identity:get_user(user_id=2377ca134c2c480fbe482fc72aa2b0d6)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:30.135640 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679fd8c6-eb32-499b-a122-111f32642bde None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:30.138189 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 86/168] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:29 2022] GET /identity/v3/users/2377ca134c2c480fbe482fc72aa2b0d6 => generated 289 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:30.139003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679fd8c6-eb32-499b-a122-111f32642bde None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:30.159752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 83/169] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:30.160864 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8042b7-f67b-492d-a38c-582de48e40e8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:30.225949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:30.239521 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:30.265301 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8042b7-f67b-492d-a38c-582de48e40e8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:30.278737 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679fd8c6-eb32-499b-a122-111f32642bde None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:30.302586 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f727eef-9bd3-44df-ae63-d130e49d88b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:30.303416 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f727eef-9bd3-44df-ae63-d130e49d88b9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:30.303702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f727eef-9bd3-44df-ae63-d130e49d88b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:30.303975 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f727eef-9bd3-44df-ae63-d130e49d88b9 None admin] PATH_INFO: `/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:30.305190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f727eef-9bd3-44df-ae63-d130e49d88b9 None admin] RBAC: Authorizing `identity:get_project(project_id=47885b7cd44548ce92d8cfb4b55c8a97)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:30.306647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f727eef-9bd3-44df-ae63-d130e49d88b9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:30.308384 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 84/170] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:30 2022] GET /identity/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:30.309152 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f727eef-9bd3-44df-ae63-d130e49d88b9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:30.338341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681ea99-36d3-4f79-bf6a-e0fb349bac9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:30.339126 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681ea99-36d3-4f79-bf6a-e0fb349bac9c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:30.339450 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681ea99-36d3-4f79-bf6a-e0fb349bac9c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:30.339729 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681ea99-36d3-4f79-bf6a-e0fb349bac9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:30.411399 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f727eef-9bd3-44df-ae63-d130e49d88b9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:30.448760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4388b-0504-4bec-8f60-48c8dda7a89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:30.449574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4388b-0504-4bec-8f60-48c8dda7a89f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:30.449861 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4388b-0504-4bec-8f60-48c8dda7a89f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:30.450137 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a4388b-0504-4bec-8f60-48c8dda7a89f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:30.462450 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a4388b-0504-4bec-8f60-48c8dda7a89f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:30.471823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-01a4388b-0504-4bec-8f60-48c8dda7a89f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 10:13:30.471823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:30.471823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:30.471823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:30.471823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:30.471823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:30.471823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:30.471823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:30.471823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:30.471823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:30.471823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:30.471823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:30.471823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:30.471823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:30.471823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:30.471823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:30.472747 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:30.472747 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:30.472747 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:30.472747 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:30.472747 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:30.472747 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:30.472747 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:30.472747 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:30.472747 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:30.472747 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:30.472747 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:30.472747 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:30.472747 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:30.472747 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:30.472747 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:30.472747 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:30.474323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:30.474323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:30.474323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:30.474323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:30.474323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:30.474323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:30.474323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:30.474323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:30.474323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:30.474323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:30.474323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 10:13:30.474323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:13:30.474323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 85/171] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jun 30 10:13:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:30.474982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4388b-0504-4bec-8f60-48c8dda7a89f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:30.485780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:30.496035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b681ea99-36d3-4f79-bf6a-e0fb349bac9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:30.497988 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b681ea99-36d3-4f79-bf6a-e0fb349bac9c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:30.572221 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:30.609057 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 87/172] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:13:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:13:30.610274 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681ea99-36d3-4f79-bf6a-e0fb349bac9c None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:30.704755 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a4388b-0504-4bec-8f60-48c8dda7a89f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:30.746947 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d358a2b9-3eb3-4272-be1d-81f91be91723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:30.748191 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358a2b9-3eb3-4272-be1d-81f91be91723 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:30.748485 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358a2b9-3eb3-4272-be1d-81f91be91723 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:30.748769 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d358a2b9-3eb3-4272-be1d-81f91be91723 None admin] PATH_INFO: `/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97/users/2377ca134c2c480fbe482fc72aa2b0d6/roles/ea0b118209d448f2902b045a6ac12919` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:30.751085 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d358a2b9-3eb3-4272-be1d-81f91be91723 None admin] RBAC: Authorizing `identity:create_grant(project_id=47885b7cd44548ce92d8cfb4b55c8a97, user_id=2377ca134c2c480fbe482fc72aa2b0d6, role_id=ea0b118209d448f2902b045a6ac12919)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:30.752828 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d358a2b9-3eb3-4272-be1d-81f91be91723 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:30.778197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681ea99-36d3-4f79-bf6a-e0fb349bac9c None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:30.818954 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 86/173] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jun 30 10:13:30 2022] PUT /identity/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97/users/2377ca134c2c480fbe482fc72aa2b0d6/roles/ea0b118209d448f2902b045a6ac12919 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:13:30.819848 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d358a2b9-3eb3-4272-be1d-81f91be91723 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:30.826002 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b6063-647a-4aaf-bc17-252b856f5345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:30.826626 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b6063-647a-4aaf-bc17-252b856f5345 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:30.826912 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b6063-647a-4aaf-bc17-252b856f5345 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:30.827189 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b6063-647a-4aaf-bc17-252b856f5345 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:30.828006 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b6063-647a-4aaf-bc17-252b856f5345 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:30.828319 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b6063-647a-4aaf-bc17-252b856f5345 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:30.829733 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b6063-647a-4aaf-bc17-252b856f5345 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:30.851976 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 88/174] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:13:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:30.860387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b6063-647a-4aaf-bc17-252b856f5345 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:30.909336 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:30.919482 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:30.956906 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d358a2b9-3eb3-4272-be1d-81f91be91723 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:31.000281 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51d50b-0d08-404d-852c-526ba4aa5771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:31.002184 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51d50b-0d08-404d-852c-526ba4aa5771 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:31.002662 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51d50b-0d08-404d-852c-526ba4aa5771 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:31.003183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51d50b-0d08-404d-852c-526ba4aa5771 None admin] PATH_INFO: `/v3/projects/ce606c0a9d6547819fdbd91ffce253f6` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:31.005343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51d50b-0d08-404d-852c-526ba4aa5771 None admin] RBAC: Authorizing `identity:get_project(project_id=ce606c0a9d6547819fdbd91ffce253f6)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:31.007038 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51d50b-0d08-404d-852c-526ba4aa5771 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:31.008983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 87/175] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:30 2022] GET /identity/v3/projects/ce606c0a9d6547819fdbd91ffce253f6 => generated 312 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:31.015711 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51d50b-0d08-404d-852c-526ba4aa5771 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:31.022289 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b6063-647a-4aaf-bc17-252b856f5345 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:31.063363 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9906e3eb-7c33-4387-adac-237302473ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:31.063363 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9906e3eb-7c33-4387-adac-237302473ac3 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:31.063363 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9906e3eb-7c33-4387-adac-237302473ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:31.063363 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9906e3eb-7c33-4387-adac-237302473ac3 None admin] PATH_INFO: `/v3/users/db5cbfa9be364e0d9c75eac6090e7967` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:31.064996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9906e3eb-7c33-4387-adac-237302473ac3 None admin] RBAC: Authorizing `identity:get_user(user_id=db5cbfa9be364e0d9c75eac6090e7967)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:31.068118 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9906e3eb-7c33-4387-adac-237302473ac3 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:31.072517 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 89/176] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:30 2022] GET /identity/v3/users/db5cbfa9be364e0d9c75eac6090e7967 => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:31.079668 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9906e3eb-7c33-4387-adac-237302473ac3 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:31.213258 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51d50b-0d08-404d-852c-526ba4aa5771 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:31.237013 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:31.243608 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9906e3eb-7c33-4387-adac-237302473ac3 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:31.275958 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:31.288688 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc825d4-42bb-4576-9b8b-e8469b121a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:31.289405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc825d4-42bb-4576-9b8b-e8469b121a17 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:31.290188 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc825d4-42bb-4576-9b8b-e8469b121a17 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:31.290188 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc825d4-42bb-4576-9b8b-e8469b121a17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:31.290812 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc825d4-42bb-4576-9b8b-e8469b121a17 None admin] RBAC: Adding query filter params (role.id=31034f95ef88402296c753c2c259f717, scope.project.id=ce606c0a9d6547819fdbd91ffce253f6, user.id=db5cbfa9be364e0d9c75eac6090e7967) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:31.291128 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc825d4-42bb-4576-9b8b-e8469b121a17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:31.293949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc825d4-42bb-4576-9b8b-e8469b121a17 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:31.305851 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9beb77-e66a-4920-9e07-865cc5a0af31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:31.306522 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9beb77-e66a-4920-9e07-865cc5a0af31 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:31.306807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9beb77-e66a-4920-9e07-865cc5a0af31 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:31.307088 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9beb77-e66a-4920-9e07-865cc5a0af31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:31.309881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 88/177] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:13:31 2022] GET /identity/v3/role_assignments?user.id=db5cbfa9be364e0d9c75eac6090e7967&scope.project.id=ce606c0a9d6547819fdbd91ffce253f6&role.id=31034f95ef88402296c753c2c259f717 => generated 608 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:31.411948 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9beb77-e66a-4920-9e07-865cc5a0af31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:31.413393 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9beb77-e66a-4920-9e07-865cc5a0af31 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:31.554834 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:31.564205 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 90/178] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:13:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 10:13:31.573298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc825d4-42bb-4576-9b8b-e8469b121a17 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:31.696286 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc825d4-42bb-4576-9b8b-e8469b121a17 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:31.736018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a4d11-61c0-41ce-9a45-ec973844d0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:31.736895 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a4d11-61c0-41ce-9a45-ec973844d0eb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:31.737191 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a4d11-61c0-41ce-9a45-ec973844d0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:31.737465 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a4d11-61c0-41ce-9a45-ec973844d0eb None admin] PATH_INFO: `/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:31.739486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a4d11-61c0-41ce-9a45-ec973844d0eb None admin] RBAC: Authorizing `identity:get_project(project_id=47885b7cd44548ce92d8cfb4b55c8a97)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:31.741195 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3a4d11-61c0-41ce-9a45-ec973844d0eb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:31.742933 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 89/179] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:31 2022] GET /identity/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:31.748943 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9beb77-e66a-4920-9e07-865cc5a0af31 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:31.879698 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9beb77-e66a-4920-9e07-865cc5a0af31 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:31.915844 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a77eb-c436-445e-b9ae-f487eec87251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:31.920022 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a77eb-c436-445e-b9ae-f487eec87251 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:31.920378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a77eb-c436-445e-b9ae-f487eec87251 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:31.920667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a77eb-c436-445e-b9ae-f487eec87251 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:31.921609 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a77eb-c436-445e-b9ae-f487eec87251 None admin] RBAC: Adding query filter params (role.id=284d1573e06d47eba0196afbe92d4d7d, scope.project.id=47885b7cd44548ce92d8cfb4b55c8a97, user.id=db5cbfa9be364e0d9c75eac6090e7967) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:31.921933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a77eb-c436-445e-b9ae-f487eec87251 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:31.923438 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a77eb-c436-445e-b9ae-f487eec87251 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:31.930665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 91/180] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:13:31 2022] GET /identity/v3/role_assignments?user.id=db5cbfa9be364e0d9c75eac6090e7967&scope.project.id=47885b7cd44548ce92d8cfb4b55c8a97&role.id=284d1573e06d47eba0196afbe92d4d7d => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:32.281217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90ac4a-2227-41a4-bbdb-1ac5ee97f178 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:32.281733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90ac4a-2227-41a4-bbdb-1ac5ee97f178 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:32.281835 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d90ac4a-2227-41a4-bbdb-1ac5ee97f178 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:32.283337 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 90/181] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:32.289174 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d048e-4794-44ae-aedc-6580ef8a79fe None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:32.289599 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d048e-4794-44ae-aedc-6580ef8a79fe None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:32.289769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86d048e-4794-44ae-aedc-6580ef8a79fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:32.307670 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-c86d048e-4794-44ae-aedc-6580ef8a79fe None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:32.308873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-c86d048e-4794-44ae-aedc-6580ef8a79fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:32.376178 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:32.388932 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 92/182] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 10:13:32.416254 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90ac4a-2227-41a4-bbdb-1ac5ee97f178 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:32.492349 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:32.544175 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d90ac4a-2227-41a4-bbdb-1ac5ee97f178 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:32.586122 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3952848-2786-4b8f-82e8-9a20234aa859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:32.587160 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3952848-2786-4b8f-82e8-9a20234aa859 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:32.587614 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3952848-2786-4b8f-82e8-9a20234aa859 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:32.588120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3952848-2786-4b8f-82e8-9a20234aa859 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:32.600059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3952848-2786-4b8f-82e8-9a20234aa859 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:32.608155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-b3952848-2786-4b8f-82e8-9a20234aa859 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:13:32.608155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:32.608155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:32.608155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:32.608155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:32.608155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:32.608155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:32.608155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:32.608155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:32.608155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:32.608155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:32.608155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:32.608155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:32.608155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:32.608155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:32.608155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:32.609219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:32.609219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:32.609219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:32.609219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:32.609219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:32.609219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:32.609219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:32.609219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:32.609219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:32.609219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:32.609219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:32.609219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:32.609219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:32.609219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:32.609219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:32.609219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:32.610213 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:32.610213 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:32.610213 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:32.610213 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:32.610213 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:32.610213 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:32.610213 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:32.610213 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:32.610213 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:32.610213 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:32.610213 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:13:32.610213 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:13:32.616654 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 91/183] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:13:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:32.618743 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d048e-4794-44ae-aedc-6580ef8a79fe None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:32.719633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86d048e-4794-44ae-aedc-6580ef8a79fe None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:32.757659 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e1068-6c68-4379-9979-748b023afc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:32.758582 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e1068-6c68-4379-9979-748b023afc14 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:32.759012 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e1068-6c68-4379-9979-748b023afc14 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:32.759488 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04e1068-6c68-4379-9979-748b023afc14 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:32.760387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e1068-6c68-4379-9979-748b023afc14 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:32.760838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e1068-6c68-4379-9979-748b023afc14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:32.762429 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04e1068-6c68-4379-9979-748b023afc14 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:32.768322 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8b716-40b7-47f4-8e19-cf55f1f2f09f None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:32.768723 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8b716-40b7-47f4-8e19-cf55f1f2f09f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:32.769101 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8b716-40b7-47f4-8e19-cf55f1f2f09f None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:32.771743 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 92/184] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:32.773032 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 93/185] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:32.776765 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed4c93-fae2-4e41-b741-4e7b08c81300 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:32.777244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed4c93-fae2-4e41-b741-4e7b08c81300 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:32.777587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ed4c93-fae2-4e41-b741-4e7b08c81300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:32.780909 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e1068-6c68-4379-9979-748b023afc14 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:32.796577 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-97ed4c93-fae2-4e41-b741-4e7b08c81300 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:32.797841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-97ed4c93-fae2-4e41-b741-4e7b08c81300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:32.845416 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:32.880152 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 93/186] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:32.900407 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04e1068-6c68-4379-9979-748b023afc14 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:32.907652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed4c93-fae2-4e41-b741-4e7b08c81300 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:32.938889 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed5773-6f16-4fbd-b337-5807b322cf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:32.939826 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed5773-6f16-4fbd-b337-5807b322cf56 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:32.940111 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed5773-6f16-4fbd-b337-5807b322cf56 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:32.940396 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed5773-6f16-4fbd-b337-5807b322cf56 None admin] PATH_INFO: `/v3/users/bf86d3102f364b04aa6239d479b8a331` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:32.956728 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ed5773-6f16-4fbd-b337-5807b322cf56 None admin] RBAC: Authorizing `identity:get_user(user_id=bf86d3102f364b04aa6239d479b8a331)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:32.958740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ed5773-6f16-4fbd-b337-5807b322cf56 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:32.960584 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 94/187] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:32 2022] GET /identity/v3/users/bf86d3102f364b04aa6239d479b8a331 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:32.965925 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed5773-6f16-4fbd-b337-5807b322cf56 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:32.972007 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:33.021706 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ed4c93-fae2-4e41-b741-4e7b08c81300 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:33.057168 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3ec49-ac6e-4b6a-8270-ec658c071cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:33.057887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3ec49-ac6e-4b6a-8270-ec658c071cdf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:33.058171 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3ec49-ac6e-4b6a-8270-ec658c071cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:33.058444 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3ec49-ac6e-4b6a-8270-ec658c071cdf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:33.059684 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d3ec49-ac6e-4b6a-8270-ec658c071cdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:33.061149 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d3ec49-ac6e-4b6a-8270-ec658c071cdf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:33.062777 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 94/188] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:13:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:33.063964 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53288799-03dd-47d5-885d-eba0e78699f2 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:33.064249 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53288799-03dd-47d5-885d-eba0e78699f2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:33.064525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53288799-03dd-47d5-885d-eba0e78699f2 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:33.067299 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 95/189] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:33.068963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53288799-03dd-47d5-885d-eba0e78699f2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:33.082168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed5773-6f16-4fbd-b337-5807b322cf56 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:33.099078 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:33.120648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef069c9-0c6a-47ed-af69-7e3931dc7ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:33.121311 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef069c9-0c6a-47ed-af69-7e3931dc7ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:33.121605 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef069c9-0c6a-47ed-af69-7e3931dc7ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:33.121886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef069c9-0c6a-47ed-af69-7e3931dc7ed7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:33.181599 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53288799-03dd-47d5-885d-eba0e78699f2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:33.192816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef069c9-0c6a-47ed-af69-7e3931dc7ed7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:33.194303 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef069c9-0c6a-47ed-af69-7e3931dc7ed7 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:33.202846 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:33.218563 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef098b06-c9d9-4124-9bd7-7cf2c43777cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:33.219233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef098b06-c9d9-4124-9bd7-7cf2c43777cc None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:33.219597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef098b06-c9d9-4124-9bd7-7cf2c43777cc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:33.219879 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef098b06-c9d9-4124-9bd7-7cf2c43777cc None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:33.220604 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef098b06-c9d9-4124-9bd7-7cf2c43777cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:33.222119 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef098b06-c9d9-4124-9bd7-7cf2c43777cc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:33.292679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 95/190] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:13:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 10:13:33.297757 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf4db3-a982-4c67-95dd-d7506e9cdac3 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:33.297757 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf4db3-a982-4c67-95dd-d7506e9cdac3 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:33.297757 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf4db3-a982-4c67-95dd-d7506e9cdac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:33.311136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-78cf4db3-a982-4c67-95dd-d7506e9cdac3 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:33.312297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-78cf4db3-a982-4c67-95dd-d7506e9cdac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:33.320171 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:33.327799 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 96/191] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jun 30 10:13:33 2022] POST /identity/v3/users => generated 297 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:13:33.336397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef098b06-c9d9-4124-9bd7-7cf2c43777cc None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:33.387610 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 96/192] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:33.389714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab846de0-2c22-4fde-8400-ae27591907a5 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:33.390277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab846de0-2c22-4fde-8400-ae27591907a5 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:33.390784 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab846de0-2c22-4fde-8400-ae27591907a5 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:33.392776 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 97/193] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:33.401830 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d69fa-b9a9-4466-9ffe-474eeda3a105 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:33.402820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d69fa-b9a9-4466-9ffe-474eeda3a105 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:33.403356 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d69fa-b9a9-4466-9ffe-474eeda3a105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:33.424944 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-fc1d69fa-b9a9-4466-9ffe-474eeda3a105 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:33.426697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-fc1d69fa-b9a9-4466-9ffe-474eeda3a105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:33.465478 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef098b06-c9d9-4124-9bd7-7cf2c43777cc None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:33.509691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a6903-9f92-412e-9ed7-ecd558fabd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:33.511129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a6903-9f92-412e-9ed7-ecd558fabd12 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:33.511129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a6903-9f92-412e-9ed7-ecd558fabd12 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:33.511322 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a6903-9f92-412e-9ed7-ecd558fabd12 None admin] PATH_INFO: `/v3/projects/95abfdd95158427e9f9f0571955df6c4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:33.511714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 98/194] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 10:13:33.512565 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d69fa-b9a9-4466-9ffe-474eeda3a105 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:33.516263 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a6903-9f92-412e-9ed7-ecd558fabd12 None admin] RBAC: Authorizing `identity:get_project(project_id=95abfdd95158427e9f9f0571955df6c4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:33.517729 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a6903-9f92-412e-9ed7-ecd558fabd12 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:33.519629 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 97/195] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:33 2022] GET /identity/v3/projects/95abfdd95158427e9f9f0571955df6c4 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:33.527190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a6903-9f92-412e-9ed7-ecd558fabd12 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:33.580531 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:33.619221 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:33.651376 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a6903-9f92-412e-9ed7-ecd558fabd12 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:33.658153 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d69fa-b9a9-4466-9ffe-474eeda3a105 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:33.691553 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b571d-6b5b-405b-bb43-b74a560b0cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:33.692316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b571d-6b5b-405b-bb43-b74a560b0cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:33.692604 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b571d-6b5b-405b-bb43-b74a560b0cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:33.692887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b571d-6b5b-405b-bb43-b74a560b0cd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:33.693797 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b571d-6b5b-405b-bb43-b74a560b0cd0 None admin] RBAC: Adding query filter params (role.id=e63bda042e104a4a93fba15ffcfe1183, scope.project.id=95abfdd95158427e9f9f0571955df6c4, user.id=bf86d3102f364b04aa6239d479b8a331) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:33.694116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b571d-6b5b-405b-bb43-b74a560b0cd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:33.696378 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b571d-6b5b-405b-bb43-b74a560b0cd0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:33.707325 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 98/196] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:13:33 2022] GET /identity/v3/role_assignments?user.id=bf86d3102f364b04aa6239d479b8a331&scope.project.id=95abfdd95158427e9f9f0571955df6c4&role.id=e63bda042e104a4a93fba15ffcfe1183 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:33.707325 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b571d-6b5b-405b-bb43-b74a560b0cd0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:33.711083 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478847b3-7b51-4f5d-b5c4-201550a76349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:33.711961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478847b3-7b51-4f5d-b5c4-201550a76349 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:33.712245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478847b3-7b51-4f5d-b5c4-201550a76349 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:33.712521 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478847b3-7b51-4f5d-b5c4-201550a76349 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:33.726441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478847b3-7b51-4f5d-b5c4-201550a76349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:33.735065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-478847b3-7b51-4f5d-b5c4-201550a76349 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 10:13:33.735065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:33.735065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:33.735065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:33.735065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:33.735065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:33.735065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:33.735065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:33.735065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:33.735065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:33.735065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:33.735065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:33.735065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:33.735065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:33.735065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:33.735065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:33.736062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:33.736062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:33.736062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:33.736062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:33.736062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:33.736062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:33.736062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:33.736062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:33.736062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:33.736062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:33.736062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:33.736062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:33.736062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:33.736062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:33.736062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:33.736062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:33.736870 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:33.736870 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:33.736870 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:33.736870 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:33.736870 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:33.736870 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:33.736870 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:33.736870 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:33.736870 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:33.736870 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:33.736870 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 10:13:33.736870 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:13:33.736870 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 99/197] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jun 30 10:13:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:33.742535 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478847b3-7b51-4f5d-b5c4-201550a76349 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:33.815375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b571d-6b5b-405b-bb43-b74a560b0cd0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:33.825577 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:33.852954 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901dc6c0-616a-4d8b-b36a-33ea6a76b3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:33.854179 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901dc6c0-616a-4d8b-b36a-33ea6a76b3c8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:33.854682 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901dc6c0-616a-4d8b-b36a-33ea6a76b3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:33.855608 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901dc6c0-616a-4d8b-b36a-33ea6a76b3c8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:33.861814 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478847b3-7b51-4f5d-b5c4-201550a76349 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:33.867301 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901dc6c0-616a-4d8b-b36a-33ea6a76b3c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:33.875476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-901dc6c0-616a-4d8b-b36a-33ea6a76b3c8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 10:13:33.875476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:33.875476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:33.875476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:33.875476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:33.875476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:33.875476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:33.875476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:33.875476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:33.875476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:33.875476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:33.875476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:33.875476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:33.875476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:33.875476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:33.875476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:33.876473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:33.876473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:33.876473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:33.876473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:33.876473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:33.876473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:33.876473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:33.876473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:33.876473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:33.876473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:33.876473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:33.876473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:33.876473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:33.876473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:33.876473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:33.876473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:33.877259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:33.877259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:33.877259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:33.877259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:33.877259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:33.877259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:33.877259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:33.877259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:33.877259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:33.877259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:33.877259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 10:13:33.877259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:13:33.877259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 99/198] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:33.882869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901dc6c0-616a-4d8b-b36a-33ea6a76b3c8 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:33.900934 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:33.907531 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f7e87-8d0d-4f4d-8e1d-163093f15bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:33.929119 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f7e87-8d0d-4f4d-8e1d-163093f15bfc None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:33.929119 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f7e87-8d0d-4f4d-8e1d-163093f15bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:33.929119 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54f7e87-8d0d-4f4d-8e1d-163093f15bfc None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:33.929119 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f7e87-8d0d-4f4d-8e1d-163093f15bfc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:33.929119 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f7e87-8d0d-4f4d-8e1d-163093f15bfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:33.929119 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54f7e87-8d0d-4f4d-8e1d-163093f15bfc None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:33.929542 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 100/199] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:13:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:33.930971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f7e87-8d0d-4f4d-8e1d-163093f15bfc None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:33.997531 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901dc6c0-616a-4d8b-b36a-33ea6a76b3c8 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:34.033676 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10790fc0-ca0f-4862-917d-42375e7e8ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:34.034448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10790fc0-ca0f-4862-917d-42375e7e8ff3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:34.034870 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10790fc0-ca0f-4862-917d-42375e7e8ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:34.035224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10790fc0-ca0f-4862-917d-42375e7e8ff3 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:34.036774 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10790fc0-ca0f-4862-917d-42375e7e8ff3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:34.037153 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10790fc0-ca0f-4862-917d-42375e7e8ff3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:34.038644 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10790fc0-ca0f-4862-917d-42375e7e8ff3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:34.045813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54f7e87-8d0d-4f4d-8e1d-163093f15bfc None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:34.047957 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 100/200] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:13:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:34.054483 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10790fc0-ca0f-4862-917d-42375e7e8ff3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:34.088144 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:34.090274 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ab686-bd55-4d39-85db-a74dbee8bdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:34.091721 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ab686-bd55-4d39-85db-a74dbee8bdf7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:34.092175 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ab686-bd55-4d39-85db-a74dbee8bdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:34.092619 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3ab686-bd55-4d39-85db-a74dbee8bdf7 None admin] PATH_INFO: `/v3/users/bf86d3102f364b04aa6239d479b8a331` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:34.095183 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ab686-bd55-4d39-85db-a74dbee8bdf7 None admin] RBAC: Authorizing `identity:get_user(user_id=bf86d3102f364b04aa6239d479b8a331)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:34.096849 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3ab686-bd55-4d39-85db-a74dbee8bdf7 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:34.111750 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 101/201] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:33 2022] GET /identity/v3/users/bf86d3102f364b04aa6239d479b8a331 => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:34.111750 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ab686-bd55-4d39-85db-a74dbee8bdf7 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:34.186141 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:34.201429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10790fc0-ca0f-4862-917d-42375e7e8ff3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:34.219741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3ab686-bd55-4d39-85db-a74dbee8bdf7 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:34.259111 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9e518-87b4-48ea-9a45-6c1b6edcad19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:34.260298 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9e518-87b4-48ea-9a45-6c1b6edcad19 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:34.260775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9e518-87b4-48ea-9a45-6c1b6edcad19 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:34.261241 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9e518-87b4-48ea-9a45-6c1b6edcad19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:34.328567 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582ec02-c292-471c-b426-036a7f8d7c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:34.329472 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582ec02-c292-471c-b426-036a7f8d7c07 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:34.329760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582ec02-c292-471c-b426-036a7f8d7c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:34.330039 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582ec02-c292-471c-b426-036a7f8d7c07 None admin] PATH_INFO: `/v3/users/2377ca134c2c480fbe482fc72aa2b0d6` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:34.335218 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe9e518-87b4-48ea-9a45-6c1b6edcad19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:34.336045 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b582ec02-c292-471c-b426-036a7f8d7c07 None admin] RBAC: Authorizing `identity:get_user(user_id=2377ca134c2c480fbe482fc72aa2b0d6)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:34.337537 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe9e518-87b4-48ea-9a45-6c1b6edcad19 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:34.337717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b582ec02-c292-471c-b426-036a7f8d7c07 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:34.339571 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 101/202] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:34 2022] GET /identity/v3/users/2377ca134c2c480fbe482fc72aa2b0d6 => generated 289 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:34.344639 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582ec02-c292-471c-b426-036a7f8d7c07 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:34.407730 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:34.447256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 102/203] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:13:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:13:34.453281 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9e518-87b4-48ea-9a45-6c1b6edcad19 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:34.453401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582ec02-c292-471c-b426-036a7f8d7c07 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:34.454823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:34.495896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d459816-eadd-4c5e-b69b-81bb0887d56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:34.496558 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d459816-eadd-4c5e-b69b-81bb0887d56b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:34.496843 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d459816-eadd-4c5e-b69b-81bb0887d56b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:34.497121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d459816-eadd-4c5e-b69b-81bb0887d56b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:34.627711 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9e518-87b4-48ea-9a45-6c1b6edcad19 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:34.656748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d459816-eadd-4c5e-b69b-81bb0887d56b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:34.658347 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d459816-eadd-4c5e-b69b-81bb0887d56b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:34.728350 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a977597-6d25-4335-bc25-b289e0d24694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:34.729550 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a977597-6d25-4335-bc25-b289e0d24694 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:34.729844 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a977597-6d25-4335-bc25-b289e0d24694 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:34.730131 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a977597-6d25-4335-bc25-b289e0d24694 None admin] PATH_INFO: `/v3/projects/95abfdd95158427e9f9f0571955df6c4` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:34.731483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a977597-6d25-4335-bc25-b289e0d24694 None admin] RBAC: Authorizing `identity:get_project(project_id=95abfdd95158427e9f9f0571955df6c4)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:34.734039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a977597-6d25-4335-bc25-b289e0d24694 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:34.736704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 103/204] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:34 2022] GET /identity/v3/projects/95abfdd95158427e9f9f0571955df6c4 => generated 302 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:34.746128 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a977597-6d25-4335-bc25-b289e0d24694 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:34.785825 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:34.790813 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 102/205] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:13:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:13:34.796723 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d459816-eadd-4c5e-b69b-81bb0887d56b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:34.815979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:34.913710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a977597-6d25-4335-bc25-b289e0d24694 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:34.959964 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2620024d-9494-43ca-8ab6-6d62a840da6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:34.961184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620024d-9494-43ca-8ab6-6d62a840da6d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:34.961736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620024d-9494-43ca-8ab6-6d62a840da6d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:34.962251 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2620024d-9494-43ca-8ab6-6d62a840da6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:34.963414 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2620024d-9494-43ca-8ab6-6d62a840da6d None admin] RBAC: Adding query filter params (role.id=284d1573e06d47eba0196afbe92d4d7d, scope.project.id=95abfdd95158427e9f9f0571955df6c4, user.id=bf86d3102f364b04aa6239d479b8a331) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:34.963936 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2620024d-9494-43ca-8ab6-6d62a840da6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:34.965867 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2620024d-9494-43ca-8ab6-6d62a840da6d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:34.974673 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 104/206] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:13:34 2022] GET /identity/v3/role_assignments?user.id=bf86d3102f364b04aa6239d479b8a331&scope.project.id=95abfdd95158427e9f9f0571955df6c4&role.id=284d1573e06d47eba0196afbe92d4d7d => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:34.996024 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d459816-eadd-4c5e-b69b-81bb0887d56b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:35.068267 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b00ec93-d651-4a68-9f3e-11bd14167bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:35.069193 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00ec93-d651-4a68-9f3e-11bd14167bd5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:35.069483 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00ec93-d651-4a68-9f3e-11bd14167bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:35.069764 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b00ec93-d651-4a68-9f3e-11bd14167bd5 None admin] PATH_INFO: `/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:35.071286 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00ec93-d651-4a68-9f3e-11bd14167bd5 None admin] RBAC: Authorizing `identity:get_project(project_id=47885b7cd44548ce92d8cfb4b55c8a97)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:35.072839 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b00ec93-d651-4a68-9f3e-11bd14167bd5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:35.075183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 103/207] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:34 2022] GET /identity/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97 => generated 298 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:35.081370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2620024d-9494-43ca-8ab6-6d62a840da6d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:35.251953 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:35.261047 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2620024d-9494-43ca-8ab6-6d62a840da6d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:35.317821 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24d5317-dcd9-4718-9867-35b98880af42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:35.318849 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d5317-dcd9-4718-9867-35b98880af42 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:35.319358 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d5317-dcd9-4718-9867-35b98880af42 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:35.319854 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24d5317-dcd9-4718-9867-35b98880af42 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:35.320884 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d5317-dcd9-4718-9867-35b98880af42 None admin] RBAC: Adding query filter params (role.id=ea0b118209d448f2902b045a6ac12919, scope.project.id=47885b7cd44548ce92d8cfb4b55c8a97, user.id=2377ca134c2c480fbe482fc72aa2b0d6) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:35.321386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d5317-dcd9-4718-9867-35b98880af42 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:35.323041 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24d5317-dcd9-4718-9867-35b98880af42 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:35.336417 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 105/208] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:13:35 2022] GET /identity/v3/role_assignments?user.id=2377ca134c2c480fbe482fc72aa2b0d6&scope.project.id=47885b7cd44548ce92d8cfb4b55c8a97&role.id=ea0b118209d448f2902b045a6ac12919 => generated 608 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:35.903632 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2c0cf-b865-4665-ad29-69e00a92544c None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:35.904123 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2c0cf-b865-4665-ad29-69e00a92544c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:35.904123 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2c0cf-b865-4665-ad29-69e00a92544c None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:35.905754 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 104/209] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:35.912505 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4573d-db36-45f8-a061-a8aac6cb5413 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:35.913153 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4573d-db36-45f8-a061-a8aac6cb5413 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:35.913623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4573d-db36-45f8-a061-a8aac6cb5413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:35.936266 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-3cf4573d-db36-45f8-a061-a8aac6cb5413 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:35.937802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-3cf4573d-db36-45f8-a061-a8aac6cb5413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:36.020359 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 106/210] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 10:13:36.046550 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2c0cf-b865-4665-ad29-69e00a92544c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:36.122095 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:36.156241 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda8166-c57b-48df-8d03-023dac98d2c7 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:36.156823 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda8166-c57b-48df-8d03-023dac98d2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:36.157294 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda8166-c57b-48df-8d03-023dac98d2c7 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:36.158579 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2c0cf-b865-4665-ad29-69e00a92544c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:36.159243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 107/211] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:36.167142 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046410a-e935-4cc3-943e-08a7e3eace4a None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:36.167473 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046410a-e935-4cc3-943e-08a7e3eace4a None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:36.171092 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046410a-e935-4cc3-943e-08a7e3eace4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:36.206528 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-c046410a-e935-4cc3-943e-08a7e3eace4a None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:36.208838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-c046410a-e935-4cc3-943e-08a7e3eace4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:36.234072 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f9cab-7971-44da-9f5d-12c42d3092c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:36.235618 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f9cab-7971-44da-9f5d-12c42d3092c4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:36.236353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f9cab-7971-44da-9f5d-12c42d3092c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:36.236819 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f9cab-7971-44da-9f5d-12c42d3092c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:36.250651 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074f9cab-7971-44da-9f5d-12c42d3092c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:36.261326 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-074f9cab-7971-44da-9f5d-12c42d3092c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:36.261326 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:36.261326 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:36.261326 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:36.261326 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:36.261326 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:36.261326 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:36.261326 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:36.261326 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:36.261326 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:36.261326 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:36.261326 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:36.261326 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:36.261326 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:36.261326 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:36.261326 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:36.262448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:36.262448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:36.262448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:36.262448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:36.262448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:36.262448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:36.262448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:36.262448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:36.262448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:36.262448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:36.262448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:36.262448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:36.262448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:36.262448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:36.262448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:36.262448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:36.263462 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:36.263462 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:36.263462 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:36.263462 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:36.263462 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:36.263462 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:36.263462 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:36.263462 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:36.263462 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:36.263462 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:36.263462 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:36.263462 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:13:36.266385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 105/212] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:36.275288 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f9cab-7971-44da-9f5d-12c42d3092c4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:36.295564 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 108/213] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:36.320232 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046410a-e935-4cc3-943e-08a7e3eace4a None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:36.339357 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:36.392060 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f9cab-7971-44da-9f5d-12c42d3092c4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:36.425715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:36.434698 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7a9e0-2ac1-48e1-a87c-ce30e654115f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:36.435500 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7a9e0-2ac1-48e1-a87c-ce30e654115f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:36.435849 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7a9e0-2ac1-48e1-a87c-ce30e654115f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:36.436239 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7a9e0-2ac1-48e1-a87c-ce30e654115f None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:36.437246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a7a9e0-2ac1-48e1-a87c-ce30e654115f None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:36.439792 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a7a9e0-2ac1-48e1-a87c-ce30e654115f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:36.439792 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a7a9e0-2ac1-48e1-a87c-ce30e654115f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:36.449117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046410a-e935-4cc3-943e-08a7e3eace4a None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:36.453872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 106/214] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:13:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:36.456500 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7a9e0-2ac1-48e1-a87c-ce30e654115f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:36.492901 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a010a4-c5d5-4be5-ac41-805fb4c76a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:36.493903 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a010a4-c5d5-4be5-ac41-805fb4c76a67 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:36.494288 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a010a4-c5d5-4be5-ac41-805fb4c76a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:36.494665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a010a4-c5d5-4be5-ac41-805fb4c76a67 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:36.496010 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a010a4-c5d5-4be5-ac41-805fb4c76a67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:36.497577 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a010a4-c5d5-4be5-ac41-805fb4c76a67 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:36.503986 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 109/215] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:13:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:36.505121 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a010a4-c5d5-4be5-ac41-805fb4c76a67 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:36.584838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7a9e0-2ac1-48e1-a87c-ce30e654115f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:36.604982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:36.620313 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86021a5c-2a28-49db-87aa-7bd2af32f770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:36.621185 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86021a5c-2a28-49db-87aa-7bd2af32f770 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:36.621466 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86021a5c-2a28-49db-87aa-7bd2af32f770 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:36.621738 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86021a5c-2a28-49db-87aa-7bd2af32f770 None admin] PATH_INFO: `/v3/users/5c9173ca54f9445b8f3c0efc45d06446` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:36.622525 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a010a4-c5d5-4be5-ac41-805fb4c76a67 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:36.636096 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86021a5c-2a28-49db-87aa-7bd2af32f770 None admin] RBAC: Authorizing `identity:get_user(user_id=5c9173ca54f9445b8f3c0efc45d06446)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:36.637567 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86021a5c-2a28-49db-87aa-7bd2af32f770 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:36.643623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 107/216] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:36 2022] GET /identity/v3/users/5c9173ca54f9445b8f3c0efc45d06446 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:36.643623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad697971-7a8a-4144-84b0-7bf430b3fd72 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:36.643623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad697971-7a8a-4144-84b0-7bf430b3fd72 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:36.643623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad697971-7a8a-4144-84b0-7bf430b3fd72 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:36.643623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 108/217] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:36.648015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad697971-7a8a-4144-84b0-7bf430b3fd72 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:36.666305 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3da93-bc3b-4e33-8d8d-7445e0920e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:36.666979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3da93-bc3b-4e33-8d8d-7445e0920e97 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:36.667293 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3da93-bc3b-4e33-8d8d-7445e0920e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:36.667566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3da93-bc3b-4e33-8d8d-7445e0920e97 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:36.668281 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3da93-bc3b-4e33-8d8d-7445e0920e97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:36.669716 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3da93-bc3b-4e33-8d8d-7445e0920e97 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:36.699137 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:36.750490 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad697971-7a8a-4144-84b0-7bf430b3fd72 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:36.785099 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c975b9-3e12-40c1-b708-43c71791811c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:36.786777 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c975b9-3e12-40c1-b708-43c71791811c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:36.787152 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c975b9-3e12-40c1-b708-43c71791811c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:36.787569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c975b9-3e12-40c1-b708-43c71791811c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:36.797092 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 110/218] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jun 30 10:13:36 2022] POST /identity/v3/users => generated 297 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:13:36.799023 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639db6b9-115b-425a-9430-f603f651b528 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:36.799468 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639db6b9-115b-425a-9430-f603f651b528 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:36.799788 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639db6b9-115b-425a-9430-f603f651b528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:36.826988 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-639db6b9-115b-425a-9430-f603f651b528 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:36.829110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-639db6b9-115b-425a-9430-f603f651b528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:36.939596 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c975b9-3e12-40c1-b708-43c71791811c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:36.943224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c975b9-3e12-40c1-b708-43c71791811c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:36.949522 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 111/219] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:36.972606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639db6b9-115b-425a-9430-f603f651b528 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:37.002804 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:37.070103 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:37.082203 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 109/220] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:13:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:13:37.088473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c975b9-3e12-40c1-b708-43c71791811c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:37.104239 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639db6b9-115b-425a-9430-f603f651b528 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:37.162502 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bec74f-0ba9-4e00-b2ae-4989fb27bfde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:37.163359 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bec74f-0ba9-4e00-b2ae-4989fb27bfde None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:37.163642 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bec74f-0ba9-4e00-b2ae-4989fb27bfde None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:37.163973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bec74f-0ba9-4e00-b2ae-4989fb27bfde None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:37.176127 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bec74f-0ba9-4e00-b2ae-4989fb27bfde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:37.184490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-b7bec74f-0ba9-4e00-b2ae-4989fb27bfde None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:13:37.184490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:37.184490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:37.184490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:37.184490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:37.184490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:37.184490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:37.184490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:37.184490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:37.184490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:37.184490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:37.184490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:37.184490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:37.184490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:37.184490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:37.184490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:37.185406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:37.185406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:37.185406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:37.185406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:37.185406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:37.185406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:37.185406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:37.185406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:37.185406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:37.185406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:37.185406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:37.185406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:37.185406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:37.185406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:37.185406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:37.185406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:37.186190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:37.186190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:37.186190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:37.186190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:37.186190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:37.186190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:37.186190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:37.186190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:37.186190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:37.186190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:37.186190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:13:37.186190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:13:37.186190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 112/221] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:13:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:37.196188 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bec74f-0ba9-4e00-b2ae-4989fb27bfde None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:37.221335 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c975b9-3e12-40c1-b708-43c71791811c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:37.268774 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c68f6-9d66-4ecc-8b65-897bf0be2616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:37.270191 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c68f6-9d66-4ecc-8b65-897bf0be2616 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:37.270477 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c68f6-9d66-4ecc-8b65-897bf0be2616 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:37.270758 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7c68f6-9d66-4ecc-8b65-897bf0be2616 None admin] PATH_INFO: `/v3/projects/95abfdd95158427e9f9f0571955df6c4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:37.274650 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7c68f6-9d66-4ecc-8b65-897bf0be2616 None admin] RBAC: Authorizing `identity:get_project(project_id=95abfdd95158427e9f9f0571955df6c4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:37.276204 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7c68f6-9d66-4ecc-8b65-897bf0be2616 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:37.278374 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 110/222] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:37 2022] GET /identity/v3/projects/95abfdd95158427e9f9f0571955df6c4 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:37.283912 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c68f6-9d66-4ecc-8b65-897bf0be2616 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:37.303757 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:37.322102 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bec74f-0ba9-4e00-b2ae-4989fb27bfde None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:37.371303 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:37.371934 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8ab91a-7a92-403c-a480-9b0b23e396fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:37.372509 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ab91a-7a92-403c-a480-9b0b23e396fd None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:37.372804 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ab91a-7a92-403c-a480-9b0b23e396fd None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:37.373086 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8ab91a-7a92-403c-a480-9b0b23e396fd None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:37.373849 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ab91a-7a92-403c-a480-9b0b23e396fd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:37.374161 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ab91a-7a92-403c-a480-9b0b23e396fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:37.376409 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8ab91a-7a92-403c-a480-9b0b23e396fd None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:37.387304 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 113/223] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:37.389085 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07fa93-134a-4ddc-9b16-0bd7a8730792 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:37.389380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07fa93-134a-4ddc-9b16-0bd7a8730792 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:37.389832 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e07fa93-134a-4ddc-9b16-0bd7a8730792 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:37.392075 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 114/224] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:37.392802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07fa93-134a-4ddc-9b16-0bd7a8730792 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:37.415840 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7c68f6-9d66-4ecc-8b65-897bf0be2616 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:37.452440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de40c472-ae0e-440c-bf04-93b2e7efe161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:37.453155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40c472-ae0e-440c-bf04-93b2e7efe161 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:37.453490 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40c472-ae0e-440c-bf04-93b2e7efe161 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:37.453817 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de40c472-ae0e-440c-bf04-93b2e7efe161 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:37.454660 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40c472-ae0e-440c-bf04-93b2e7efe161 None admin] RBAC: Adding query filter params (role.id=31034f95ef88402296c753c2c259f717, scope.project.id=95abfdd95158427e9f9f0571955df6c4, user.id=5c9173ca54f9445b8f3c0efc45d06446) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:37.455025 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40c472-ae0e-440c-bf04-93b2e7efe161 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:37.456571 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de40c472-ae0e-440c-bf04-93b2e7efe161 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:37.462741 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 111/225] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:13:37 2022] GET /identity/v3/role_assignments?user.id=5c9173ca54f9445b8f3c0efc45d06446&scope.project.id=95abfdd95158427e9f9f0571955df6c4&role.id=31034f95ef88402296c753c2c259f717 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:37.473371 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629d6dd-a0f0-41de-b031-27c3901251d1 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:37.473655 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629d6dd-a0f0-41de-b031-27c3901251d1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:37.473851 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629d6dd-a0f0-41de-b031-27c3901251d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:37.506370 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5629d6dd-a0f0-41de-b031-27c3901251d1 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:37.507795 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5629d6dd-a0f0-41de-b031-27c3901251d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:37.516047 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e07fa93-134a-4ddc-9b16-0bd7a8730792 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:37.555505 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78ca03-4411-46d1-9116-3cc3442ab0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:37.556589 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78ca03-4411-46d1-9116-3cc3442ab0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:37.556966 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78ca03-4411-46d1-9116-3cc3442ab0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:37.557357 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f78ca03-4411-46d1-9116-3cc3442ab0d7 None admin] PATH_INFO: `/v3/users/bf86d3102f364b04aa6239d479b8a331` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:37.558710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78ca03-4411-46d1-9116-3cc3442ab0d7 None admin] RBAC: Authorizing `identity:get_user(user_id=bf86d3102f364b04aa6239d479b8a331)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:37.560359 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f78ca03-4411-46d1-9116-3cc3442ab0d7 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:37.562219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 115/226] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:37 2022] GET /identity/v3/users/bf86d3102f364b04aa6239d479b8a331 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:37.568429 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78ca03-4411-46d1-9116-3cc3442ab0d7 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:37.653623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 112/227] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:37.666705 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:37.708555 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629d6dd-a0f0-41de-b031-27c3901251d1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:37.718062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f78ca03-4411-46d1-9116-3cc3442ab0d7 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:37.722151 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:37.788566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc2c77-2a2b-4cf8-9b07-39cd3fecca60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:37.789360 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc2c77-2a2b-4cf8-9b07-39cd3fecca60 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:37.789648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc2c77-2a2b-4cf8-9b07-39cd3fecca60 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:37.789928 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdc2c77-2a2b-4cf8-9b07-39cd3fecca60 None admin] PATH_INFO: `/v3/projects/95abfdd95158427e9f9f0571955df6c4` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:37.792161 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc2c77-2a2b-4cf8-9b07-39cd3fecca60 None admin] RBAC: Authorizing `identity:get_project(project_id=95abfdd95158427e9f9f0571955df6c4)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:37.793622 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdc2c77-2a2b-4cf8-9b07-39cd3fecca60 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:37.795672 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 116/228] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:37 2022] GET /identity/v3/projects/95abfdd95158427e9f9f0571955df6c4 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:37.803018 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc2c77-2a2b-4cf8-9b07-39cd3fecca60 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:37.860821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5629d6dd-a0f0-41de-b031-27c3901251d1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:37.901199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa07490-d22a-44af-95ed-ca9f33f6207d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:37.902415 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa07490-d22a-44af-95ed-ca9f33f6207d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:37.902938 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa07490-d22a-44af-95ed-ca9f33f6207d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:37.903463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa07490-d22a-44af-95ed-ca9f33f6207d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:37.915533 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa07490-d22a-44af-95ed-ca9f33f6207d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:37.926887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-3fa07490-d22a-44af-95ed-ca9f33f6207d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 10:13:37.926887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:37.926887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:37.926887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:37.926887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:37.926887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:37.926887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:37.926887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:37.926887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:37.926887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:37.926887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:37.926887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:37.926887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:37.926887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:37.926887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:37.926887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:37.928129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:37.928129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:37.928129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:37.928129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:37.928129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:37.928129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:37.928129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:37.928129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:37.928129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:37.928129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:37.928129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:37.928129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:37.928129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:37.928129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:37.928129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:37.928129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:37.928932 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:37.928932 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:37.928932 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:37.928932 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:37.928932 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:37.928932 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:37.928932 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:37.928932 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:37.928932 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:37.928932 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:37.928932 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 10:13:37.928932 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:13:37.931711 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 113/229] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jun 30 10:13:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:13:37.937317 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa07490-d22a-44af-95ed-ca9f33f6207d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:38.012493 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdc2c77-2a2b-4cf8-9b07-39cd3fecca60 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:38.062589 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:38.069017 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8119a7-e34e-4ca1-88da-e0193a7b1ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:38.070186 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8119a7-e34e-4ca1-88da-e0193a7b1ed4 None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:38.070671 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8119a7-e34e-4ca1-88da-e0193a7b1ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:38.071197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8119a7-e34e-4ca1-88da-e0193a7b1ed4 None admin] PATH_INFO: `/v3/projects/95abfdd95158427e9f9f0571955df6c4/users/bf86d3102f364b04aa6239d479b8a331/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:38.073513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8119a7-e34e-4ca1-88da-e0193a7b1ed4 None admin] RBAC: Authorizing `identity:create_grant(project_id=95abfdd95158427e9f9f0571955df6c4, user_id=bf86d3102f364b04aa6239d479b8a331, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:38.073643 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:38.075333 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8119a7-e34e-4ca1-88da-e0193a7b1ed4 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:38.113986 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa07490-d22a-44af-95ed-ca9f33f6207d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:38.149977 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b89b11-414b-4807-8c9d-a644d64f4395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:38.150662 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b89b11-414b-4807-8c9d-a644d64f4395 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:38.150948 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b89b11-414b-4807-8c9d-a644d64f4395 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:38.151227 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b89b11-414b-4807-8c9d-a644d64f4395 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:38.152052 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b89b11-414b-4807-8c9d-a644d64f4395 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:38.152367 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b89b11-414b-4807-8c9d-a644d64f4395 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:38.153838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b89b11-414b-4807-8c9d-a644d64f4395 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:38.162581 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 117/230] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jun 30 10:13:37 2022] PUT /identity/v3/projects/95abfdd95158427e9f9f0571955df6c4/users/bf86d3102f364b04aa6239d479b8a331/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:13:38.174873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 114/231] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:13:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:38.181041 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8119a7-e34e-4ca1-88da-e0193a7b1ed4 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:38.317470 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8119a7-e34e-4ca1-88da-e0193a7b1ed4 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:38.366813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc5f33-a2bf-4ea8-83e1-5a3d05c3fd64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:38.367809 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc5f33-a2bf-4ea8-83e1-5a3d05c3fd64 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:38.368097 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc5f33-a2bf-4ea8-83e1-5a3d05c3fd64 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:38.368375 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc5f33-a2bf-4ea8-83e1-5a3d05c3fd64 None admin] PATH_INFO: `/v3/users/bf86d3102f364b04aa6239d479b8a331` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:38.370391 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc5f33-a2bf-4ea8-83e1-5a3d05c3fd64 None admin] RBAC: Authorizing `identity:get_user(user_id=bf86d3102f364b04aa6239d479b8a331)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:38.371903 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc5f33-a2bf-4ea8-83e1-5a3d05c3fd64 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:38.373820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 118/232] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:38 2022] GET /identity/v3/users/bf86d3102f364b04aa6239d479b8a331 => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:38.380087 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b89b11-414b-4807-8c9d-a644d64f4395 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:38.507795 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b89b11-414b-4807-8c9d-a644d64f4395 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:38.555700 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:38.569323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e3926e-8c76-40a5-990c-efb138c30777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:38.570148 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3926e-8c76-40a5-990c-efb138c30777 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:38.570638 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3926e-8c76-40a5-990c-efb138c30777 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:38.571115 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e3926e-8c76-40a5-990c-efb138c30777 None admin] PATH_INFO: `/v3/projects/95abfdd95158427e9f9f0571955df6c4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:38.572533 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e3926e-8c76-40a5-990c-efb138c30777 None admin] RBAC: Authorizing `identity:get_project(project_id=95abfdd95158427e9f9f0571955df6c4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:38.574214 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e3926e-8c76-40a5-990c-efb138c30777 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:38.576454 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 115/233] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:38 2022] GET /identity/v3/projects/95abfdd95158427e9f9f0571955df6c4 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:38.582748 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc5f33-a2bf-4ea8-83e1-5a3d05c3fd64 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:38.664454 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:38.727494 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc5f33-a2bf-4ea8-83e1-5a3d05c3fd64 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:38.767311 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b263d-b88c-4831-9ed2-7ea09fa65568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:38.768298 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b263d-b88c-4831-9ed2-7ea09fa65568 None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:38.768585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b263d-b88c-4831-9ed2-7ea09fa65568 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:38.768914 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b263d-b88c-4831-9ed2-7ea09fa65568 None admin] PATH_INFO: `/v3/projects/95abfdd95158427e9f9f0571955df6c4/users/bf86d3102f364b04aa6239d479b8a331/roles/284d1573e06d47eba0196afbe92d4d7d` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:38.773622 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488b263d-b88c-4831-9ed2-7ea09fa65568 None admin] RBAC: Authorizing `identity:create_grant(project_id=95abfdd95158427e9f9f0571955df6c4, user_id=bf86d3102f364b04aa6239d479b8a331, role_id=284d1573e06d47eba0196afbe92d4d7d)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:38.775195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488b263d-b88c-4831-9ed2-7ea09fa65568 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:38.836855 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 119/234] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jun 30 10:13:38 2022] PUT /identity/v3/projects/95abfdd95158427e9f9f0571955df6c4/users/bf86d3102f364b04aa6239d479b8a331/roles/284d1573e06d47eba0196afbe92d4d7d => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:13:39.546814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a78471-4620-4871-b5a4-470d17d05280 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:39.547347 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a78471-4620-4871-b5a4-470d17d05280 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:39.547457 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a78471-4620-4871-b5a4-470d17d05280 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:39.548923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 116/235] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:39.555779 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b0b6b-96a7-480b-99a1-436973b5e899 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:39.555779 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b0b6b-96a7-480b-99a1-436973b5e899 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:39.555779 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b0b6b-96a7-480b-99a1-436973b5e899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:39.583504 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-d19b0b6b-96a7-480b-99a1-436973b5e899 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:39.585090 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d19b0b6b-96a7-480b-99a1-436973b5e899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:39.659856 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb651154-ebc9-4f14-824b-24be411d0dbe None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:39.660696 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb651154-ebc9-4f14-824b-24be411d0dbe None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:39.661046 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb651154-ebc9-4f14-824b-24be411d0dbe None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:39.662720 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 117/236] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:39.671857 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600abb1-bfee-47e5-9ac8-ddf2ed09097a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:39.672140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600abb1-bfee-47e5-9ac8-ddf2ed09097a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:39.672417 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600abb1-bfee-47e5-9ac8-ddf2ed09097a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:39.700284 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 120/237] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 10:13:39.704310 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d600abb1-bfee-47e5-9ac8-ddf2ed09097a None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:39.705529 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d600abb1-bfee-47e5-9ac8-ddf2ed09097a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:39.727657 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b0b6b-96a7-480b-99a1-436973b5e899 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:39.822732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:39.828183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 118/238] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 10:13:39.829394 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d3de4-32c2-43f2-90d6-d7038e764107 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:39.835598 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d3de4-32c2-43f2-90d6-d7038e764107 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:39.843187 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85d3de4-32c2-43f2-90d6-d7038e764107 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:39.843187 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 119/239] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:39 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:39.843602 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f7df5-2de0-48f1-8458-eeae9e5dc6e5 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:39.843956 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f7df5-2de0-48f1-8458-eeae9e5dc6e5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:39.844299 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f7df5-2de0-48f1-8458-eeae9e5dc6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:39.874097 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0c6f7df5-2de0-48f1-8458-eeae9e5dc6e5 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:39.875296 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0c6f7df5-2de0-48f1-8458-eeae9e5dc6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:39.884806 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b0b6b-96a7-480b-99a1-436973b5e899 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:39.932857 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d2178-0d31-4448-9a61-dd785878286f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:39.933661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d2178-0d31-4448-9a61-dd785878286f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:39.933946 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d2178-0d31-4448-9a61-dd785878286f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:39.934221 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35d2178-0d31-4448-9a61-dd785878286f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:39.935433 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35d2178-0d31-4448-9a61-dd785878286f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:39.936899 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35d2178-0d31-4448-9a61-dd785878286f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:39.938740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 121/240] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:13:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:39.939316 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d2178-0d31-4448-9a61-dd785878286f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:39.978004 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 120/241] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 10:13:39.978864 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f7df5-2de0-48f1-8458-eeae9e5dc6e5 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:40.047424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:40.076156 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35d2178-0d31-4448-9a61-dd785878286f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:40.093677 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:40.094719 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f7df5-2de0-48f1-8458-eeae9e5dc6e5 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:40.114309 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f5114-1056-4760-b9ac-1cdb46f646a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:40.115790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f5114-1056-4760-b9ac-1cdb46f646a6 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:40.116429 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f5114-1056-4760-b9ac-1cdb46f646a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:40.117150 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9f5114-1056-4760-b9ac-1cdb46f646a6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:40.131859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9f5114-1056-4760-b9ac-1cdb46f646a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:40.133758 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c3fa8-e47e-4060-b233-231f49c413cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:40.134751 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c3fa8-e47e-4060-b233-231f49c413cb None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:40.135216 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c3fa8-e47e-4060-b233-231f49c413cb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:40.135728 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c3fa8-e47e-4060-b233-231f49c413cb None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:40.136638 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c3fa8-e47e-4060-b233-231f49c413cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:40.138379 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c3fa8-e47e-4060-b233-231f49c413cb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:40.140719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-ba9f5114-1056-4760-b9ac-1cdb46f646a6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 10:13:40.140719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:40.140719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:40.140719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:40.140719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:40.140719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:40.140719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:40.140719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:40.140719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:40.140719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:40.140719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:40.140719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:40.140719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:40.140719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:40.140719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:40.140719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:40.141778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:40.141778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:40.141778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:40.141778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:40.141778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:40.141778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:40.141778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:40.141778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:40.141778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:40.141778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:40.141778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:40.141778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:40.141778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:40.141778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:40.141778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:40.141778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:40.142749 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:40.142749 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:40.142749 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:40.142749 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:40.142749 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:40.142749 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:40.142749 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:40.142749 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:40.142749 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:40.142749 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:40.142749 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 10:13:40.142749 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:13:40.144012 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 122/242] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:40.150401 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f5114-1056-4760-b9ac-1cdb46f646a6 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:40.227354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 121/243] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jun 30 10:13:39 2022] POST /identity/v3/users => generated 293 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:13:40.233485 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c3fa8-e47e-4060-b233-231f49c413cb None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:40.294039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9f5114-1056-4760-b9ac-1cdb46f646a6 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:40.367061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e27e94-eb28-4284-91fc-c75d22f04bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:40.368948 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e27e94-eb28-4284-91fc-c75d22f04bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:40.370248 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e27e94-eb28-4284-91fc-c75d22f04bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:40.371606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e27e94-eb28-4284-91fc-c75d22f04bd7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:40.395793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e27e94-eb28-4284-91fc-c75d22f04bd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:40.406243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-71e27e94-eb28-4284-91fc-c75d22f04bd7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:40.406243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:40.406243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:40.406243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:40.406243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:40.406243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:40.406243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:40.406243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:40.406243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:40.406243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:40.406243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:40.406243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:40.406243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:40.406243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:40.406243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:40.406243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:40.407380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:40.407380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:40.407380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:40.407380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:40.407380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:40.407380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:40.407380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:40.407380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:40.407380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:40.407380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:40.407380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:40.407380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:40.407380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:40.407380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:40.407380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:40.407380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:40.408361 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:40.408361 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:40.408361 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:40.408361 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:40.408361 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:40.408361 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:40.408361 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:40.408361 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:40.408361 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:40.408361 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:40.408361 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:40.408361 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:13:40.415077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 123/244] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:40.421393 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e27e94-eb28-4284-91fc-c75d22f04bd7 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:40.455400 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c3fa8-e47e-4060-b233-231f49c413cb None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:40.480543 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:40.486511 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:40.513134 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b29262-5972-458a-8d90-5bb4837f56eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:40.514234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b29262-5972-458a-8d90-5bb4837f56eb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:40.514624 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b29262-5972-458a-8d90-5bb4837f56eb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:40.515255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b29262-5972-458a-8d90-5bb4837f56eb None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:40.516564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b29262-5972-458a-8d90-5bb4837f56eb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:40.517186 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b29262-5972-458a-8d90-5bb4837f56eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:40.520111 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b29262-5972-458a-8d90-5bb4837f56eb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:40.536033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 122/245] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:13:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:40.539958 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b29262-5972-458a-8d90-5bb4837f56eb None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:40.576782 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e27e94-eb28-4284-91fc-c75d22f04bd7 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:40.612702 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266242d-dabc-45be-ba93-0a6ef0b99cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:40.614192 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266242d-dabc-45be-ba93-0a6ef0b99cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:40.614587 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266242d-dabc-45be-ba93-0a6ef0b99cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:40.614873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266242d-dabc-45be-ba93-0a6ef0b99cd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:40.615661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266242d-dabc-45be-ba93-0a6ef0b99cd3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:40.615969 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266242d-dabc-45be-ba93-0a6ef0b99cd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:40.617557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266242d-dabc-45be-ba93-0a6ef0b99cd3 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:40.626054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 124/246] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:13:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:40.632289 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266242d-dabc-45be-ba93-0a6ef0b99cd3 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:40.653372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b29262-5972-458a-8d90-5bb4837f56eb None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:40.702672 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4611c74-f113-415a-9f59-27049347c41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:40.703950 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4611c74-f113-415a-9f59-27049347c41c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:40.704404 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4611c74-f113-415a-9f59-27049347c41c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:40.704868 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4611c74-f113-415a-9f59-27049347c41c None admin] PATH_INFO: `/v3/users/af6d27a7a2a24f11ad70678a3d01a87e` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:40.726870 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4611c74-f113-415a-9f59-27049347c41c None admin] RBAC: Authorizing `identity:get_user(user_id=af6d27a7a2a24f11ad70678a3d01a87e)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:40.728715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4611c74-f113-415a-9f59-27049347c41c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:40.730724 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 123/247] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:40 2022] GET /identity/v3/users/af6d27a7a2a24f11ad70678a3d01a87e => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:40.736167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4611c74-f113-415a-9f59-27049347c41c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:40.752958 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:40.759437 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266242d-dabc-45be-ba93-0a6ef0b99cd3 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:40.784257 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:40.808563 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d516963-20f3-429c-80cf-f3ebd80ad23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:40.810849 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d516963-20f3-429c-80cf-f3ebd80ad23c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:40.811763 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d516963-20f3-429c-80cf-f3ebd80ad23c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:40.812284 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d516963-20f3-429c-80cf-f3ebd80ad23c None admin] PATH_INFO: `/v3/users/5c9173ca54f9445b8f3c0efc45d06446` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:40.815606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d516963-20f3-429c-80cf-f3ebd80ad23c None admin] RBAC: Authorizing `identity:get_user(user_id=5c9173ca54f9445b8f3c0efc45d06446)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:40.819088 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d516963-20f3-429c-80cf-f3ebd80ad23c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:40.822072 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 125/248] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:40 2022] GET /identity/v3/users/5c9173ca54f9445b8f3c0efc45d06446 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:40.823422 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcf664-f4af-45f2-b0c5-0715e9cf0d0f None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:40.824264 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcf664-f4af-45f2-b0c5-0715e9cf0d0f None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:40.824546 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dcf664-f4af-45f2-b0c5-0715e9cf0d0f None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:40.826112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 126/249] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:40.832416 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcf664-f4af-45f2-b0c5-0715e9cf0d0f None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:40.874964 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4611c74-f113-415a-9f59-27049347c41c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:40.921611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88e15b-3fbb-4d0e-964a-9e7ff1f65b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:40.922549 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88e15b-3fbb-4d0e-964a-9e7ff1f65b6e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:40.922835 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88e15b-3fbb-4d0e-964a-9e7ff1f65b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:40.923104 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88e15b-3fbb-4d0e-964a-9e7ff1f65b6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:40.963957 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dcf664-f4af-45f2-b0c5-0715e9cf0d0f None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:41.015373 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88e15b-3fbb-4d0e-964a-9e7ff1f65b6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:41.015373 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88e15b-3fbb-4d0e-964a-9e7ff1f65b6e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:41.024854 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af34567-b0ff-4a72-8ed9-80ae4905a9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:41.025733 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af34567-b0ff-4a72-8ed9-80ae4905a9c8 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:41.026015 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af34567-b0ff-4a72-8ed9-80ae4905a9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:41.026288 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af34567-b0ff-4a72-8ed9-80ae4905a9c8 None admin] PATH_INFO: `/v3/projects/95abfdd95158427e9f9f0571955df6c4` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:41.027572 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af34567-b0ff-4a72-8ed9-80ae4905a9c8 None admin] RBAC: Authorizing `identity:get_project(project_id=95abfdd95158427e9f9f0571955df6c4)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:41.029021 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af34567-b0ff-4a72-8ed9-80ae4905a9c8 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:41.030940 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 127/250] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:40 2022] GET /identity/v3/projects/95abfdd95158427e9f9f0571955df6c4 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:41.031904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6bf1b-c7ed-4c34-ae34-cfb331dfbc73 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:41.032188 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6bf1b-c7ed-4c34-ae34-cfb331dfbc73 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:41.032464 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b6bf1b-c7ed-4c34-ae34-cfb331dfbc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:41.060803 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-b1b6bf1b-c7ed-4c34-ae34-cfb331dfbc73 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:41.062000 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-b1b6bf1b-c7ed-4c34-ae34-cfb331dfbc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:41.080320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:41.120530 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:41.192435 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 124/251] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:13:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:13:41.194465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 128/252] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:41.195121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88e15b-3fbb-4d0e-964a-9e7ff1f65b6e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:41.204329 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6bf1b-c7ed-4c34-ae34-cfb331dfbc73 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:41.331737 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88e15b-3fbb-4d0e-964a-9e7ff1f65b6e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:41.398722 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b6bf1b-c7ed-4c34-ae34-cfb331dfbc73 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:41.403793 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f11ec1-35fa-420a-89be-588d89ac3ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:41.404854 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f11ec1-35fa-420a-89be-588d89ac3ab2 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:41.405477 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f11ec1-35fa-420a-89be-588d89ac3ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:41.405911 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f11ec1-35fa-420a-89be-588d89ac3ab2 None admin] PATH_INFO: `/v3/projects/95abfdd95158427e9f9f0571955df6c4/users/5c9173ca54f9445b8f3c0efc45d06446/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:41.413178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f11ec1-35fa-420a-89be-588d89ac3ab2 None admin] RBAC: Authorizing `identity:create_grant(project_id=95abfdd95158427e9f9f0571955df6c4, user_id=5c9173ca54f9445b8f3c0efc45d06446, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:41.414816 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f11ec1-35fa-420a-89be-588d89ac3ab2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:41.417964 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:41.448150 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38a3da-aff0-4058-a26c-c1710ce11b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:41.449319 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38a3da-aff0-4058-a26c-c1710ce11b1e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:41.449758 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38a3da-aff0-4058-a26c-c1710ce11b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:41.450180 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c38a3da-aff0-4058-a26c-c1710ce11b1e None admin] PATH_INFO: `/v3/projects/95abfdd95158427e9f9f0571955df6c4` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:41.451523 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c38a3da-aff0-4058-a26c-c1710ce11b1e None admin] RBAC: Authorizing `identity:get_project(project_id=95abfdd95158427e9f9f0571955df6c4)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:41.453184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c38a3da-aff0-4058-a26c-c1710ce11b1e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:41.455360 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 129/253] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:41 2022] GET /identity/v3/projects/95abfdd95158427e9f9f0571955df6c4 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:41.456269 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38a3da-aff0-4058-a26c-c1710ce11b1e None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:41.496222 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:41.540938 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 125/254] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jun 30 10:13:41 2022] PUT /identity/v3/projects/95abfdd95158427e9f9f0571955df6c4/users/5c9173ca54f9445b8f3c0efc45d06446/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:13:41.542482 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b9617-ca8d-47d3-ba85-ba1685fe1a42 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:41.542841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b9617-ca8d-47d3-ba85-ba1685fe1a42 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:41.543247 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60b9617-ca8d-47d3-ba85-ba1685fe1a42 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:41.544871 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 126/255] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:41.549247 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b9617-ca8d-47d3-ba85-ba1685fe1a42 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:41.584859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c38a3da-aff0-4058-a26c-c1710ce11b1e None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:41.656233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d23b2c-25fc-4719-b895-debb74694f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:41.658937 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d23b2c-25fc-4719-b895-debb74694f98 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:41.660971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d23b2c-25fc-4719-b895-debb74694f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:41.661493 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d23b2c-25fc-4719-b895-debb74694f98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:41.693297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d23b2c-25fc-4719-b895-debb74694f98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:41.708372 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-51d23b2c-25fc-4719-b895-debb74694f98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:13:41.708372 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:41.708372 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:41.708372 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:41.708372 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:41.708372 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:41.708372 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:41.708372 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:41.708372 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:41.708372 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:41.708372 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:41.708372 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:41.708372 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:41.708372 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:41.708372 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:41.708372 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:41.709299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:41.709299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:41.709299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:41.709299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:41.709299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:41.709299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:41.709299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:41.709299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:41.709299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:41.709299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:41.709299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:41.709299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:41.709299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:41.709299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:41.709299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:41.709299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:41.710089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:41.710089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:41.710089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:41.710089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:41.710089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:41.710089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:41.710089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:41.710089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:41.710089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:41.710089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:41.710089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:13:41.710089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:13:41.716702 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 130/256] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:13:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:41.721022 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214f063-c557-48c0-8c7d-47fabb203b16 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:41.721509 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214f063-c557-48c0-8c7d-47fabb203b16 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:41.721924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3214f063-c557-48c0-8c7d-47fabb203b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:41.750296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-3214f063-c557-48c0-8c7d-47fabb203b16 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:41.751627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-3214f063-c557-48c0-8c7d-47fabb203b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:41.758035 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60b9617-ca8d-47d3-ba85-ba1685fe1a42 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:41.805920 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381456e-e945-4aac-9715-6d2dbeeadb79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:41.806935 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381456e-e945-4aac-9715-6d2dbeeadb79 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:41.807460 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381456e-e945-4aac-9715-6d2dbeeadb79 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:41.807930 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a381456e-e945-4aac-9715-6d2dbeeadb79 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:41.808959 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a381456e-e945-4aac-9715-6d2dbeeadb79 None admin] RBAC: Adding query filter params (role.id=ea0b118209d448f2902b045a6ac12919, scope.project.id=95abfdd95158427e9f9f0571955df6c4, user.id=af6d27a7a2a24f11ad70678a3d01a87e) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:41.809978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a381456e-e945-4aac-9715-6d2dbeeadb79 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:41.812113 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a381456e-e945-4aac-9715-6d2dbeeadb79 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:41.820831 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 127/257] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:13:41 2022] GET /identity/v3/role_assignments?user.id=af6d27a7a2a24f11ad70678a3d01a87e&scope.project.id=95abfdd95158427e9f9f0571955df6c4&role.id=ea0b118209d448f2902b045a6ac12919 => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:41.822000 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381456e-e945-4aac-9715-6d2dbeeadb79 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:41.865817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 131/258] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:41.898270 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:41.899737 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214f063-c557-48c0-8c7d-47fabb203b16 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:41.970512 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a381456e-e945-4aac-9715-6d2dbeeadb79 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:41.975780 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:42.027306 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f7f80-dd78-4831-a24b-de681f86056c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:42.028350 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f7f80-dd78-4831-a24b-de681f86056c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:42.028827 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f7f80-dd78-4831-a24b-de681f86056c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:42.029327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68f7f80-dd78-4831-a24b-de681f86056c None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:42.030900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f7f80-dd78-4831-a24b-de681f86056c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:42.031549 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f7f80-dd78-4831-a24b-de681f86056c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:42.033222 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68f7f80-dd78-4831-a24b-de681f86056c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:42.054674 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 128/259] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:42.059822 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f7f80-dd78-4831-a24b-de681f86056c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:42.178807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3214f063-c557-48c0-8c7d-47fabb203b16 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:42.246006 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee6397-a465-46f7-bae3-996211ea060e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:42.246748 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee6397-a465-46f7-bae3-996211ea060e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:42.247037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee6397-a465-46f7-bae3-996211ea060e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:42.247338 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee6397-a465-46f7-bae3-996211ea060e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:42.263161 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68f7f80-dd78-4831-a24b-de681f86056c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:42.270830 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee6397-a465-46f7-bae3-996211ea060e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:42.279163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-abee6397-a465-46f7-bae3-996211ea060e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 10:13:42.279163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:42.279163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:42.279163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:42.279163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:42.279163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:42.279163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:42.279163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:42.279163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:42.279163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:42.279163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:42.279163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:42.279163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:42.279163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:42.279163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:42.279163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:42.280061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:42.280061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:42.280061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:42.280061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:42.280061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:42.280061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:42.280061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:42.280061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:42.280061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:42.280061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:42.280061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:42.280061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:42.280061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:42.280061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:42.280061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:42.280061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:42.280926 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:42.280926 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:42.280926 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:42.280926 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:42.280926 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:42.280926 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:42.280926 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:42.280926 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:42.280926 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:42.280926 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:42.280926 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 10:13:42.280926 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:13:42.295430 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 132/260] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jun 30 10:13:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:42.296357 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee6397-a465-46f7-bae3-996211ea060e None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:42.300181 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:42.323088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69cade3-613c-425e-ac5c-876fa4331668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:42.324275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cade3-613c-425e-ac5c-876fa4331668 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:42.324702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cade3-613c-425e-ac5c-876fa4331668 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:42.325146 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cade3-613c-425e-ac5c-876fa4331668 None admin] PATH_INFO: `/v3/users/bf86d3102f364b04aa6239d479b8a331` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:42.327396 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69cade3-613c-425e-ac5c-876fa4331668 None admin] RBAC: Authorizing `identity:get_user(user_id=bf86d3102f364b04aa6239d479b8a331)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:42.329022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69cade3-613c-425e-ac5c-876fa4331668 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:42.335297 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 129/261] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:42 2022] GET /identity/v3/users/bf86d3102f364b04aa6239d479b8a331 => generated 283 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:42.340507 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69cade3-613c-425e-ac5c-876fa4331668 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:42.436919 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:42.483846 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69cade3-613c-425e-ac5c-876fa4331668 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:42.487942 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee6397-a465-46f7-bae3-996211ea060e None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:42.530811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecba2c9-cff7-4ff0-a1bf-4d4154537076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:42.532170 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecba2c9-cff7-4ff0-a1bf-4d4154537076 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:42.532722 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecba2c9-cff7-4ff0-a1bf-4d4154537076 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:42.533142 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecba2c9-cff7-4ff0-a1bf-4d4154537076 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:42.582039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17333191-b6c7-433b-9774-da1df1cf9606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:42.582856 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17333191-b6c7-433b-9774-da1df1cf9606 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:42.583207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17333191-b6c7-433b-9774-da1df1cf9606 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:42.583541 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17333191-b6c7-433b-9774-da1df1cf9606 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:42.584307 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17333191-b6c7-433b-9774-da1df1cf9606 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:42.584619 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17333191-b6c7-433b-9774-da1df1cf9606 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:42.586029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17333191-b6c7-433b-9774-da1df1cf9606 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:42.601533 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:42.604547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 133/262] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:13:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:42.609990 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17333191-b6c7-433b-9774-da1df1cf9606 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:42.615778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecba2c9-cff7-4ff0-a1bf-4d4154537076 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:42.618455 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecba2c9-cff7-4ff0-a1bf-4d4154537076 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:42.850918 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 130/263] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:13:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:13:42.857932 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecba2c9-cff7-4ff0-a1bf-4d4154537076 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:42.858647 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17333191-b6c7-433b-9774-da1df1cf9606 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:42.952617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813f920-f5ac-41ce-be23-1240a8b4aa91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:42.953505 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813f920-f5ac-41ce-be23-1240a8b4aa91 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:42.953794 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813f920-f5ac-41ce-be23-1240a8b4aa91 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:42.954077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c813f920-f5ac-41ce-be23-1240a8b4aa91 None admin] PATH_INFO: `/v3/users/bf86d3102f364b04aa6239d479b8a331` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:42.960006 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813f920-f5ac-41ce-be23-1240a8b4aa91 None admin] RBAC: Authorizing `identity:get_user(user_id=bf86d3102f364b04aa6239d479b8a331)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:42.961472 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c813f920-f5ac-41ce-be23-1240a8b4aa91 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:42.963133 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 134/264] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:42 2022] GET /identity/v3/users/bf86d3102f364b04aa6239d479b8a331 => generated 283 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:42.968460 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813f920-f5ac-41ce-be23-1240a8b4aa91 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:43.001864 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:43.047636 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:43.087214 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecba2c9-cff7-4ff0-a1bf-4d4154537076 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:43.172000 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62dc74d-97e4-4895-9d9d-84f73122b0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:43.172896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62dc74d-97e4-4895-9d9d-84f73122b0ec None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:43.173183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62dc74d-97e4-4895-9d9d-84f73122b0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:43.173464 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62dc74d-97e4-4895-9d9d-84f73122b0ec None admin] PATH_INFO: `/v3/projects/95abfdd95158427e9f9f0571955df6c4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:43.174717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62dc74d-97e4-4895-9d9d-84f73122b0ec None admin] RBAC: Authorizing `identity:get_project(project_id=95abfdd95158427e9f9f0571955df6c4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:43.176978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62dc74d-97e4-4895-9d9d-84f73122b0ec None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:43.178755 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 131/265] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:42 2022] GET /identity/v3/projects/95abfdd95158427e9f9f0571955df6c4 => generated 302 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:43.188017 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62dc74d-97e4-4895-9d9d-84f73122b0ec None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:43.245422 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c813f920-f5ac-41ce-be23-1240a8b4aa91 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:43.311956 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd289ad-e6e1-4778-b3fa-30986d8f7bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:43.312983 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd289ad-e6e1-4778-b3fa-30986d8f7bf8 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:43.313492 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd289ad-e6e1-4778-b3fa-30986d8f7bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:43.313966 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd289ad-e6e1-4778-b3fa-30986d8f7bf8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:43.400997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62dc74d-97e4-4895-9d9d-84f73122b0ec None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:43.403363 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd289ad-e6e1-4778-b3fa-30986d8f7bf8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:43.405048 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd289ad-e6e1-4778-b3fa-30986d8f7bf8 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:43.419577 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:43.442247 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b858f93-eb0e-4bfe-a4a3-45abc606feba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:43.442247 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b858f93-eb0e-4bfe-a4a3-45abc606feba None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:43.442247 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b858f93-eb0e-4bfe-a4a3-45abc606feba None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:43.442247 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b858f93-eb0e-4bfe-a4a3-45abc606feba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:43.442661 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b858f93-eb0e-4bfe-a4a3-45abc606feba None admin] RBAC: Adding query filter params (role.id=e63bda042e104a4a93fba15ffcfe1183, scope.project.id=95abfdd95158427e9f9f0571955df6c4, user.id=bf86d3102f364b04aa6239d479b8a331) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:43.442911 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b858f93-eb0e-4bfe-a4a3-45abc606feba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:43.444654 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b858f93-eb0e-4bfe-a4a3-45abc606feba None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:43.451994 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 132/266] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:13:43 2022] GET /identity/v3/role_assignments?user.id=bf86d3102f364b04aa6239d479b8a331&scope.project.id=95abfdd95158427e9f9f0571955df6c4&role.id=e63bda042e104a4a93fba15ffcfe1183 => generated 608 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:43.453643 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fb2f1-ccc6-4c0c-9dc0-5f86eaaf612c None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:43.454524 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fb2f1-ccc6-4c0c-9dc0-5f86eaaf612c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:43.455235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633fb2f1-ccc6-4c0c-9dc0-5f86eaaf612c None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:43.457539 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 133/267] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:43.459542 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44fa13-b6ed-4f89-9a26-56fe4b1cb0bc None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:43.460040 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44fa13-b6ed-4f89-9a26-56fe4b1cb0bc None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:43.460492 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44fa13-b6ed-4f89-9a26-56fe4b1cb0bc None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:43.462119 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 134/268] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:43.464146 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24664bf-9f4e-4460-ab89-a3d8e1595a10 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:43.464897 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24664bf-9f4e-4460-ab89-a3d8e1595a10 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:43.465441 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24664bf-9f4e-4460-ab89-a3d8e1595a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:43.489974 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c24664bf-9f4e-4460-ab89-a3d8e1595a10 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:43.492726 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c24664bf-9f4e-4460-ab89-a3d8e1595a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:43.515804 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:43.584934 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 135/269] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:43.587075 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09244a-00fd-41ec-be40-deaf337ffce2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:43.587616 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09244a-00fd-41ec-be40-deaf337ffce2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:43.588139 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a09244a-00fd-41ec-be40-deaf337ffce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:43.594735 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 135/270] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:13:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 627 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 10:13:43.601206 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd289ad-e6e1-4778-b3fa-30986d8f7bf8 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:43.610535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1a09244a-00fd-41ec-be40-deaf337ffce2 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:43.612119 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1a09244a-00fd-41ec-be40-deaf337ffce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:43.697373 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 136/271] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:43.697984 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09244a-00fd-41ec-be40-deaf337ffce2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:43.781036 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd289ad-e6e1-4778-b3fa-30986d8f7bf8 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:43.796899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:43.800780 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:43.818737 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a09244a-00fd-41ec-be40-deaf337ffce2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:43.828507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25afff7-9ca3-4be4-b497-c395f24a63dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:43.829752 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25afff7-9ca3-4be4-b497-c395f24a63dd None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:43.830042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25afff7-9ca3-4be4-b497-c395f24a63dd None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:43.830625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25afff7-9ca3-4be4-b497-c395f24a63dd None admin] PATH_INFO: `/v3/projects/95abfdd95158427e9f9f0571955df6c4` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:43.832713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25afff7-9ca3-4be4-b497-c395f24a63dd None admin] RBAC: Authorizing `identity:get_project(project_id=95abfdd95158427e9f9f0571955df6c4)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:43.834351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25afff7-9ca3-4be4-b497-c395f24a63dd None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:43.837266 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 136/272] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:43 2022] GET /identity/v3/projects/95abfdd95158427e9f9f0571955df6c4 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:43.838359 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25afff7-9ca3-4be4-b497-c395f24a63dd None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:43.858630 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96efcedf-2a4b-411d-a238-13840ae3341d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:43.859601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efcedf-2a4b-411d-a238-13840ae3341d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:43.859927 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efcedf-2a4b-411d-a238-13840ae3341d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:43.860177 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efcedf-2a4b-411d-a238-13840ae3341d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:43.861788 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96efcedf-2a4b-411d-a238-13840ae3341d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:43.863294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96efcedf-2a4b-411d-a238-13840ae3341d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:43.864955 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 137/273] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:13:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:43.865810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96efcedf-2a4b-411d-a238-13840ae3341d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:43.977107 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25afff7-9ca3-4be4-b497-c395f24a63dd None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:43.995869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96efcedf-2a4b-411d-a238-13840ae3341d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:44.012949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8883d797-f260-45be-8915-b9724927e2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:44.013908 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883d797-f260-45be-8915-b9724927e2f5 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:44.014196 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883d797-f260-45be-8915-b9724927e2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:44.014567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8883d797-f260-45be-8915-b9724927e2f5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:44.026478 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8883d797-f260-45be-8915-b9724927e2f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:44.033974 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e1cda-3143-41cd-8cb7-fa68f8afb558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:44.034447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-8883d797-f260-45be-8915-b9724927e2f5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:44.034447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:44.034447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:44.034447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:44.034447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:44.034447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:44.034447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:44.034447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:44.034447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:44.034447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:44.034447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:44.034447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:44.034447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:44.034447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:44.034447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:44.034447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:44.036073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:44.036073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:44.036073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:44.036073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:44.036073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:44.036073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:44.036073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:44.036073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:44.036073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:44.036073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:44.036073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:44.036073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:44.036073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:44.036073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:44.036073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:44.036073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:44.036971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:44.036971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:44.036971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:44.036971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:44.036971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:44.036971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:44.036971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:44.036971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:44.036971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:44.036971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:44.036971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:44.036971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:13:44.037566 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e1cda-3143-41cd-8cb7-fa68f8afb558 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:44.037566 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e1cda-3143-41cd-8cb7-fa68f8afb558 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:44.037566 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024e1cda-3143-41cd-8cb7-fa68f8afb558 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:44.037566 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e1cda-3143-41cd-8cb7-fa68f8afb558 None admin] RBAC: Adding query filter params (role.id=284d1573e06d47eba0196afbe92d4d7d, scope.project.id=95abfdd95158427e9f9f0571955df6c4, user.id=bf86d3102f364b04aa6239d479b8a331) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:44.037799 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 137/274] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:44.037863 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e1cda-3143-41cd-8cb7-fa68f8afb558 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:44.037927 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8883d797-f260-45be-8915-b9724927e2f5 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:44.038692 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024e1cda-3143-41cd-8cb7-fa68f8afb558 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:44.045310 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 138/275] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:13:43 2022] GET /identity/v3/role_assignments?user.id=bf86d3102f364b04aa6239d479b8a331&scope.project.id=95abfdd95158427e9f9f0571955df6c4&role.id=284d1573e06d47eba0196afbe92d4d7d => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:44.046126 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e1cda-3143-41cd-8cb7-fa68f8afb558 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:44.076084 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:44.091944 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:44.157382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8883d797-f260-45be-8915-b9724927e2f5 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:44.202308 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024e1cda-3143-41cd-8cb7-fa68f8afb558 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:44.240476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e403235-d1c2-4e85-b744-985981c0c904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:44.241087 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e403235-d1c2-4e85-b744-985981c0c904 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:44.241373 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e403235-d1c2-4e85-b744-985981c0c904 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:44.241657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e403235-d1c2-4e85-b744-985981c0c904 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:44.242427 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e403235-d1c2-4e85-b744-985981c0c904 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:44.242742 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e403235-d1c2-4e85-b744-985981c0c904 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:44.244218 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e403235-d1c2-4e85-b744-985981c0c904 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:44.250779 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a098b4c8-030c-4306-8fee-c346e8467dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:44.251500 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098b4c8-030c-4306-8fee-c346e8467dca None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:44.251793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098b4c8-030c-4306-8fee-c346e8467dca None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:44.252065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a098b4c8-030c-4306-8fee-c346e8467dca None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:44.252794 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098b4c8-030c-4306-8fee-c346e8467dca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:44.254529 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 139/276] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:13:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:44.255444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a098b4c8-030c-4306-8fee-c346e8467dca None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:44.260244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e403235-d1c2-4e85-b744-985981c0c904 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:44.344427 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:44.357048 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 138/277] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jun 30 10:13:44 2022] POST /identity/v3/users => generated 293 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:13:44.376112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e403235-d1c2-4e85-b744-985981c0c904 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:44.419098 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f8953-2d87-4993-a499-35a9566b93fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:44.420412 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f8953-2d87-4993-a499-35a9566b93fc None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:44.420939 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f8953-2d87-4993-a499-35a9566b93fc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:44.421567 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f8953-2d87-4993-a499-35a9566b93fc None admin] PATH_INFO: `/v3/users/a525304a661b4698b194e55640479bfd` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:44.429563 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2f753-dd60-4b0f-9b26-c06ac9dbaebf None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:44.429875 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2f753-dd60-4b0f-9b26-c06ac9dbaebf None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:44.430145 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2f753-dd60-4b0f-9b26-c06ac9dbaebf None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:44.431949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 139/278] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:44.437006 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b9651-4851-427c-8088-879d5763e8d8 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:44.437297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b9651-4851-427c-8088-879d5763e8d8 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:44.437641 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b9651-4851-427c-8088-879d5763e8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:44.442031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f8953-2d87-4993-a499-35a9566b93fc None admin] RBAC: Authorizing `identity:get_user(user_id=a525304a661b4698b194e55640479bfd)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:44.445405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161f8953-2d87-4993-a499-35a9566b93fc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:44.447640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 140/279] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:44 2022] GET /identity/v3/users/a525304a661b4698b194e55640479bfd => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:44.456550 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f8953-2d87-4993-a499-35a9566b93fc None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:44.468417 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-a04b9651-4851-427c-8088-879d5763e8d8 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:44.469661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-a04b9651-4851-427c-8088-879d5763e8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:44.560879 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 140/280] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:44.585000 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b9651-4851-427c-8088-879d5763e8d8 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:44.591438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f8953-2d87-4993-a499-35a9566b93fc None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:44.614410 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:44.624096 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:44.630410 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd9f4aa-c747-44bb-9fd7-f3cbd76a1918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:44.631563 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9f4aa-c747-44bb-9fd7-f3cbd76a1918 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:44.631851 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9f4aa-c747-44bb-9fd7-f3cbd76a1918 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:44.634087 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd9f4aa-c747-44bb-9fd7-f3cbd76a1918 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:44.634740 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9f4aa-c747-44bb-9fd7-f3cbd76a1918 None admin] RBAC: Adding query filter params (role.id=31034f95ef88402296c753c2c259f717, user.id=a525304a661b4698b194e55640479bfd, scope.system=all) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:44.635066 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9f4aa-c747-44bb-9fd7-f3cbd76a1918 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:44.636564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd9f4aa-c747-44bb-9fd7-f3cbd76a1918 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:44.642573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 141/281] 10.222.0.47 () {56 vars in 1308 bytes} [Thu Jun 30 10:13:44 2022] GET /identity/v3/role_assignments?user.id=a525304a661b4698b194e55640479bfd&scope.system=all&role.id=31034f95ef88402296c753c2c259f717 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:44.708601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da672b03-3fa7-4fbe-9c17-487a1b873a4b None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:44.708975 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da672b03-3fa7-4fbe-9c17-487a1b873a4b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:44.709260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da672b03-3fa7-4fbe-9c17-487a1b873a4b None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:44.710847 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 142/282] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:44.714719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b9651-4851-427c-8088-879d5763e8d8 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:44.717919 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90a8cd-af14-4836-84ac-3abac31354e0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:44.718201 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90a8cd-af14-4836-84ac-3abac31354e0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:44.718484 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90a8cd-af14-4836-84ac-3abac31354e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:44.743213 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-2a90a8cd-af14-4836-84ac-3abac31354e0 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:44.744507 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2a90a8cd-af14-4836-84ac-3abac31354e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:44.765906 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bab89-defa-4845-80e4-0d6119860d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:44.766662 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bab89-defa-4845-80e4-0d6119860d94 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:44.766951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bab89-defa-4845-80e4-0d6119860d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:44.767230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03bab89-defa-4845-80e4-0d6119860d94 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:44.784374 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03bab89-defa-4845-80e4-0d6119860d94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:44.795629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-f03bab89-defa-4845-80e4-0d6119860d94 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:44.795629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:44.795629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:44.795629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:44.795629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:44.795629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:44.795629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:44.795629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:44.795629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:44.795629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:44.795629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:44.795629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:44.795629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:44.795629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:44.795629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:44.795629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:44.796583 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:44.796583 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:44.796583 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:44.796583 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:44.796583 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:44.796583 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:44.796583 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:44.796583 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:44.796583 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:44.796583 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:44.796583 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:44.796583 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:44.796583 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:44.796583 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:44.796583 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:44.796583 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:44.797351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:44.797351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:44.797351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:44.797351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:44.797351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:44.797351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:44.797351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:44.797351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:44.797351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:44.797351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:44.797351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:44.797351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:13:44.797351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 141/283] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:44.805347 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bab89-defa-4845-80e4-0d6119860d94 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:44.835661 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 143/284] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:44.859838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a90a8cd-af14-4836-84ac-3abac31354e0 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:44.919458 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:44.950224 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03bab89-defa-4845-80e4-0d6119860d94 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:45.010769 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a90a8cd-af14-4836-84ac-3abac31354e0 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:45.014795 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f880134-c571-4e39-9339-8bf1e01b4dfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:45.016346 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f880134-c571-4e39-9339-8bf1e01b4dfd None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:45.016985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f880134-c571-4e39-9339-8bf1e01b4dfd None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:45.017402 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f880134-c571-4e39-9339-8bf1e01b4dfd None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:45.021847 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f880134-c571-4e39-9339-8bf1e01b4dfd None admin] RBAC: Adding query filter params (name=member) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:45.022334 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f880134-c571-4e39-9339-8bf1e01b4dfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:45.023990 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f880134-c571-4e39-9339-8bf1e01b4dfd None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:45.038676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 142/285] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:13:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:45.045202 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f880134-c571-4e39-9339-8bf1e01b4dfd None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:45.067323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:45.084169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd858e0-85b8-4936-b04e-0218be76d132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:45.085275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd858e0-85b8-4936-b04e-0218be76d132 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:45.085741 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd858e0-85b8-4936-b04e-0218be76d132 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:45.086159 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd858e0-85b8-4936-b04e-0218be76d132 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:45.102925 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd858e0-85b8-4936-b04e-0218be76d132 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:45.117537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-bfd858e0-85b8-4936-b04e-0218be76d132 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 10:13:45.117537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:45.117537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:45.117537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:45.117537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:45.117537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:45.117537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:45.117537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:45.117537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:45.117537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:45.117537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:45.117537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:45.117537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:45.117537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:45.117537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:45.117537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:45.118652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:45.118652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:45.118652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:45.118652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:45.118652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:45.118652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:45.118652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:45.118652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:45.118652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:45.118652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:45.118652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:45.118652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:45.118652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:45.118652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:45.118652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:45.118652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:45.119691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:45.119691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:45.119691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:45.119691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:45.119691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:45.119691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:45.119691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:45.119691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:45.119691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:45.119691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:45.119691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 10:13:45.119691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:13:45.127280 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 144/286] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:13:45.132473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd858e0-85b8-4936-b04e-0218be76d132 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:45.229176 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f880134-c571-4e39-9339-8bf1e01b4dfd None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:45.287315 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd858e0-85b8-4936-b04e-0218be76d132 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:45.321464 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f8f2b-be0c-4996-8a32-755a2c75ce56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:45.322126 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f8f2b-be0c-4996-8a32-755a2c75ce56 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:45.322408 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f8f2b-be0c-4996-8a32-755a2c75ce56 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:45.322649 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba0755-a6cf-4a42-93e7-5b2c64881951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:45.322772 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f8f2b-be0c-4996-8a32-755a2c75ce56 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:45.323439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f8f2b-be0c-4996-8a32-755a2c75ce56 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:45.323747 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f8f2b-be0c-4996-8a32-755a2c75ce56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:45.324086 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba0755-a6cf-4a42-93e7-5b2c64881951 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:45.324600 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba0755-a6cf-4a42-93e7-5b2c64881951 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:45.325012 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba0755-a6cf-4a42-93e7-5b2c64881951 None admin] PATH_INFO: `/v3/users/5c9173ca54f9445b8f3c0efc45d06446` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:45.325119 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f8f2b-be0c-4996-8a32-755a2c75ce56 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:45.329573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba0755-a6cf-4a42-93e7-5b2c64881951 None admin] RBAC: Authorizing `identity:get_user(user_id=5c9173ca54f9445b8f3c0efc45d06446)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:45.331234 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ba0755-a6cf-4a42-93e7-5b2c64881951 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:45.334411 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 143/287] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:45 2022] GET /identity/v3/users/5c9173ca54f9445b8f3c0efc45d06446 => generated 297 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:45.339633 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 145/288] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:13:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:45.344520 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba0755-a6cf-4a42-93e7-5b2c64881951 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:45.345063 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f8f2b-be0c-4996-8a32-755a2c75ce56 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:45.369441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:45.456654 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:45.499953 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ba0755-a6cf-4a42-93e7-5b2c64881951 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:45.556293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f8f2b-be0c-4996-8a32-755a2c75ce56 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:45.568807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbb329-6ee2-49de-99ac-1705bee8c34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:45.569466 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbb329-6ee2-49de-99ac-1705bee8c34c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:45.569751 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbb329-6ee2-49de-99ac-1705bee8c34c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:45.570031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbb329-6ee2-49de-99ac-1705bee8c34c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:45.598023 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456dbea-828d-40c6-8b10-65747294c9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:45.599059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456dbea-828d-40c6-8b10-65747294c9fb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:45.599369 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456dbea-828d-40c6-8b10-65747294c9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:45.599650 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c456dbea-828d-40c6-8b10-65747294c9fb None admin] PATH_INFO: `/v3/users/af6d27a7a2a24f11ad70678a3d01a87e` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:45.603020 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c456dbea-828d-40c6-8b10-65747294c9fb None admin] RBAC: Authorizing `identity:get_user(user_id=af6d27a7a2a24f11ad70678a3d01a87e)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:45.604503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c456dbea-828d-40c6-8b10-65747294c9fb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:45.606934 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 146/289] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:45 2022] GET /identity/v3/users/af6d27a7a2a24f11ad70678a3d01a87e => generated 297 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:45.612897 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456dbea-828d-40c6-8b10-65747294c9fb None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:45.709807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bbb329-6ee2-49de-99ac-1705bee8c34c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:45.711360 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bbb329-6ee2-49de-99ac-1705bee8c34c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:45.760632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:45.787867 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c456dbea-828d-40c6-8b10-65747294c9fb None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:45.817941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:45.852914 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118a892-d4c5-4d5c-88e9-c41a01b6e2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:45.853788 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118a892-d4c5-4d5c-88e9-c41a01b6e2ba None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:45.854073 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118a892-d4c5-4d5c-88e9-c41a01b6e2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:45.854353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8118a892-d4c5-4d5c-88e9-c41a01b6e2ba None admin] PATH_INFO: `/v3/projects/95abfdd95158427e9f9f0571955df6c4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:45.860001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8118a892-d4c5-4d5c-88e9-c41a01b6e2ba None admin] RBAC: Authorizing `identity:get_project(project_id=95abfdd95158427e9f9f0571955df6c4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:45.861470 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8118a892-d4c5-4d5c-88e9-c41a01b6e2ba None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:45.865045 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 147/290] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:45 2022] GET /identity/v3/projects/95abfdd95158427e9f9f0571955df6c4 => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:45.870539 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118a892-d4c5-4d5c-88e9-c41a01b6e2ba None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:45.917638 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 144/291] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:13:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:13:45.923433 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbb329-6ee2-49de-99ac-1705bee8c34c None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:45.983525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8118a892-d4c5-4d5c-88e9-c41a01b6e2ba None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:46.022817 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6acc1-b0e8-40b2-aba3-5614e1c478a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:46.023740 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6acc1-b0e8-40b2-aba3-5614e1c478a0 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:46.024026 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6acc1-b0e8-40b2-aba3-5614e1c478a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:46.024302 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e6acc1-b0e8-40b2-aba3-5614e1c478a0 None admin] PATH_INFO: `/v3/projects/95abfdd95158427e9f9f0571955df6c4/users/af6d27a7a2a24f11ad70678a3d01a87e/roles/ea0b118209d448f2902b045a6ac12919` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:46.026285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6acc1-b0e8-40b2-aba3-5614e1c478a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=95abfdd95158427e9f9f0571955df6c4, user_id=af6d27a7a2a24f11ad70678a3d01a87e, role_id=ea0b118209d448f2902b045a6ac12919)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:46.027788 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e6acc1-b0e8-40b2-aba3-5614e1c478a0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:46.116869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbb329-6ee2-49de-99ac-1705bee8c34c None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:46.134384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:46.135803 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 148/292] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jun 30 10:13:45 2022] PUT /identity/v3/projects/95abfdd95158427e9f9f0571955df6c4/users/af6d27a7a2a24f11ad70678a3d01a87e/roles/ea0b118209d448f2902b045a6ac12919 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:13:46.162581 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8952186-1de3-4448-b666-4a8ce869f871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:46.163598 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8952186-1de3-4448-b666-4a8ce869f871 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:46.163943 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8952186-1de3-4448-b666-4a8ce869f871 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:46.164254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8952186-1de3-4448-b666-4a8ce869f871 None admin] PATH_INFO: `/v3/projects/95abfdd95158427e9f9f0571955df6c4` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:46.165652 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8952186-1de3-4448-b666-4a8ce869f871 None admin] RBAC: Authorizing `identity:get_project(project_id=95abfdd95158427e9f9f0571955df6c4)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:46.167614 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8952186-1de3-4448-b666-4a8ce869f871 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:46.171194 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 145/293] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:45 2022] GET /identity/v3/projects/95abfdd95158427e9f9f0571955df6c4 => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:46.178215 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6acc1-b0e8-40b2-aba3-5614e1c478a0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:46.245940 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:46.337136 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e6acc1-b0e8-40b2-aba3-5614e1c478a0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:46.381988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d2dff-ba51-4c44-a334-47ba8e2cbe47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:46.382642 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d2dff-ba51-4c44-a334-47ba8e2cbe47 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:46.382938 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d2dff-ba51-4c44-a334-47ba8e2cbe47 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:46.383227 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d2dff-ba51-4c44-a334-47ba8e2cbe47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:46.384146 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d2dff-ba51-4c44-a334-47ba8e2cbe47 None admin] RBAC: Adding query filter params (role.id=31034f95ef88402296c753c2c259f717, scope.project.id=95abfdd95158427e9f9f0571955df6c4, user.id=5c9173ca54f9445b8f3c0efc45d06446) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:46.384463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d2dff-ba51-4c44-a334-47ba8e2cbe47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:46.385928 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d2dff-ba51-4c44-a334-47ba8e2cbe47 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:46.395067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 149/294] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:13:46 2022] GET /identity/v3/role_assignments?user.id=5c9173ca54f9445b8f3c0efc45d06446&scope.project.id=95abfdd95158427e9f9f0571955df6c4&role.id=31034f95ef88402296c753c2c259f717 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:46.916206 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af170b-e4ae-4704-ba2e-baf184d98c89 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:46.917252 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af170b-e4ae-4704-ba2e-baf184d98c89 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:46.917778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af170b-e4ae-4704-ba2e-baf184d98c89 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:46.919681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 146/295] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:46.925182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c82d3d-7f1f-4b34-9748-edbff726ad8e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:46.925768 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c82d3d-7f1f-4b34-9748-edbff726ad8e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:46.925883 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c82d3d-7f1f-4b34-9748-edbff726ad8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:46.946540 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e8c82d3d-7f1f-4b34-9748-edbff726ad8e None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:46.948106 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e8c82d3d-7f1f-4b34-9748-edbff726ad8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:47.017468 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045a86c-8431-4083-85df-85a252d7ce03 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:47.017774 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045a86c-8431-4083-85df-85a252d7ce03 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:47.018035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045a86c-8431-4083-85df-85a252d7ce03 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:47.020175 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 147/296] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:13:47.026094 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6bcfe-641a-44e9-aa09-3469267d5010 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:47.026094 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6bcfe-641a-44e9-aa09-3469267d5010 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:47.026309 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc6bcfe-641a-44e9-aa09-3469267d5010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:47.028680 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 150/297] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:47.045469 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-fcc6bcfe-641a-44e9-aa09-3469267d5010 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:47.046645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-fcc6bcfe-641a-44e9-aa09-3469267d5010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:47.057101 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c82d3d-7f1f-4b34-9748-edbff726ad8e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:47.112287 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:47.143377 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 148/298] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:47.160510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6bcfe-641a-44e9-aa09-3469267d5010 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:47.186168 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c82d3d-7f1f-4b34-9748-edbff726ad8e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:47.220162 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f590e-5557-4c37-b446-191c34ff0c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:47.220961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f590e-5557-4c37-b446-191c34ff0c77 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:47.221252 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f590e-5557-4c37-b446-191c34ff0c77 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:47.221533 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f590e-5557-4c37-b446-191c34ff0c77 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:47.232663 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f590e-5557-4c37-b446-191c34ff0c77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:47.240993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-556f590e-5557-4c37-b446-191c34ff0c77 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 10:13:47.240993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:47.240993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:47.240993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:47.240993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:47.240993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:47.240993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:47.240993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:47.240993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:47.240993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:47.240993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:47.240993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:47.240993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:47.240993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:47.240993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:47.240993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:47.241876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:47.241876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:47.241876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:47.241876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:47.241876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:47.241876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:47.241876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:47.241876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:47.241876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:47.241876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:47.241876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:47.241876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:47.241876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:47.241876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:47.241876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:47.241876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:47.243572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:47.243572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:47.243572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:47.243572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:47.243572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:47.243572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:47.243572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:47.243572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:47.243572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:47.243572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:47.243572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 10:13:47.243572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:13:47.243572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 151/299] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:13:47.243572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7763b-ec84-494b-9bfd-cfef9ddb1c37 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:47.244376 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7763b-ec84-494b-9bfd-cfef9ddb1c37 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:47.244376 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d7763b-ec84-494b-9bfd-cfef9ddb1c37 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:47.245437 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 152/300] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:47.249998 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7763b-ec84-494b-9bfd-cfef9ddb1c37 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:47.275965 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc6bcfe-641a-44e9-aa09-3469267d5010 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:47.291900 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:47.312896 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8833fa-e26c-4c30-8d80-e7ec091ad0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:47.313693 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8833fa-e26c-4c30-8d80-e7ec091ad0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:47.313978 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8833fa-e26c-4c30-8d80-e7ec091ad0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:47.314290 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8833fa-e26c-4c30-8d80-e7ec091ad0d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:47.315759 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8833fa-e26c-4c30-8d80-e7ec091ad0d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:47.317483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8833fa-e26c-4c30-8d80-e7ec091ad0d7 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:47.319066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 149/301] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:13:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:47.323728 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30071ab5-8f98-4833-81e3-45c1da0e317c None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:47.323728 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30071ab5-8f98-4833-81e3-45c1da0e317c None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:47.323728 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30071ab5-8f98-4833-81e3-45c1da0e317c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:47.337310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-30071ab5-8f98-4833-81e3-45c1da0e317c None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:47.338530 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-30071ab5-8f98-4833-81e3-45c1da0e317c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:47.364096 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:47.369696 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d7763b-ec84-494b-9bfd-cfef9ddb1c37 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:47.407074 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 150/302] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:47.408263 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f4792-ccd8-46a8-b853-35792912353f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:47.408979 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f4792-ccd8-46a8-b853-35792912353f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:47.409267 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f4792-ccd8-46a8-b853-35792912353f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:47.409550 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f4792-ccd8-46a8-b853-35792912353f None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:47.410975 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f4792-ccd8-46a8-b853-35792912353f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:47.411453 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f4792-ccd8-46a8-b853-35792912353f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:47.412859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f4792-ccd8-46a8-b853-35792912353f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:47.420148 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30071ab5-8f98-4833-81e3-45c1da0e317c None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:47.427029 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 153/303] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:13:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:47.428154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f4792-ccd8-46a8-b853-35792912353f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:47.536980 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30071ab5-8f98-4833-81e3-45c1da0e317c None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:47.541248 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f4792-ccd8-46a8-b853-35792912353f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:47.551485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:47.573323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d1639-cc5b-48dd-8b3d-6a9cca9eed2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:47.574000 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d1639-cc5b-48dd-8b3d-6a9cca9eed2b None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:47.574423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d1639-cc5b-48dd-8b3d-6a9cca9eed2b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:47.574575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d1639-cc5b-48dd-8b3d-6a9cca9eed2b None admin] PATH_INFO: `/v3/groups` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:47.575356 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d1639-cc5b-48dd-8b3d-6a9cca9eed2b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:47.576771 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d1639-cc5b-48dd-8b3d-6a9cca9eed2b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:47.580067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba4b88-0466-4ec8-8e08-a08637b09f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:47.580946 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba4b88-0466-4ec8-8e08-a08637b09f08 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:47.581227 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba4b88-0466-4ec8-8e08-a08637b09f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:47.581499 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba4b88-0466-4ec8-8e08-a08637b09f08 None admin] PATH_INFO: `/v3/users/461011f415a8438dbe71bfc473184cf7` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:47.595757 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba4b88-0466-4ec8-8e08-a08637b09f08 None admin] RBAC: Authorizing `identity:get_user(user_id=461011f415a8438dbe71bfc473184cf7)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:47.597298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba4b88-0466-4ec8-8e08-a08637b09f08 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:47.599061 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 154/304] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:47 2022] GET /identity/v3/users/461011f415a8438dbe71bfc473184cf7 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:47.599856 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba4b88-0466-4ec8-8e08-a08637b09f08 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:47.618502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:47.648703 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 151/305] 10.222.0.47 () {60 vars in 1134 bytes} [Thu Jun 30 10:13:47 2022] POST /identity/v3/groups => generated 229 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:13:47.649516 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d1639-cc5b-48dd-8b3d-6a9cca9eed2b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:47.733933 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba4b88-0466-4ec8-8e08-a08637b09f08 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:47.782247 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba46aa2-d391-4b11-86f4-5e9119566475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:47.783010 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba46aa2-d391-4b11-86f4-5e9119566475 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:47.783318 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba46aa2-d391-4b11-86f4-5e9119566475 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:47.783611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba46aa2-d391-4b11-86f4-5e9119566475 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:47.799457 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba46aa2-d391-4b11-86f4-5e9119566475 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:47.804471 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d1639-cc5b-48dd-8b3d-6a9cca9eed2b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:47.809328 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-eba46aa2-d391-4b11-86f4-5e9119566475 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:47.809328 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:47.809328 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:47.809328 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:47.809328 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:47.809328 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:47.809328 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:47.809328 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:47.809328 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:47.809328 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:47.809328 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:47.809328 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:47.809328 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:47.809328 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:47.809328 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:47.809328 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:47.810294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:47.810294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:47.810294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:47.810294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:47.810294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:47.810294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:47.810294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:47.810294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:47.810294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:47.810294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:47.810294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:47.810294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:47.810294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:47.810294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:47.810294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:47.810294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:47.811068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:47.811068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:47.811068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:47.811068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:47.811068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:47.811068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:47.811068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:47.811068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:47.811068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:47.811068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:47.811068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:47.811068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:13:47.811068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 155/306] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:47.818426 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba46aa2-d391-4b11-86f4-5e9119566475 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:47.858381 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af8980-c143-4c79-a47a-55f1bb851152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:47.859053 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af8980-c143-4c79-a47a-55f1bb851152 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:47.859371 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af8980-c143-4c79-a47a-55f1bb851152 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:47.859675 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4af8980-c143-4c79-a47a-55f1bb851152 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:47.860644 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af8980-c143-4c79-a47a-55f1bb851152 None admin] RBAC: Adding query filter params (role.id=ea0b118209d448f2902b045a6ac12919, user.id=461011f415a8438dbe71bfc473184cf7, scope.system=all) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:47.861034 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af8980-c143-4c79-a47a-55f1bb851152 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:47.862705 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4af8980-c143-4c79-a47a-55f1bb851152 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:47.872572 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 152/307] 10.222.0.47 () {56 vars in 1308 bytes} [Thu Jun 30 10:13:47 2022] GET /identity/v3/role_assignments?user.id=461011f415a8438dbe71bfc473184cf7&scope.system=all&role.id=ea0b118209d448f2902b045a6ac12919 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:47.922052 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:47.945842 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba46aa2-d391-4b11-86f4-5e9119566475 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:47.984542 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfdbee5-6c55-49ba-ad9a-ce20674464be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:47.985235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdbee5-6c55-49ba-ad9a-ce20674464be None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:47.985520 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdbee5-6c55-49ba-ad9a-ce20674464be None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:47.985800 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdbee5-6c55-49ba-ad9a-ce20674464be None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:47.986572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfdbee5-6c55-49ba-ad9a-ce20674464be None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:47.986883 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfdbee5-6c55-49ba-ad9a-ce20674464be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:47.988990 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfdbee5-6c55-49ba-ad9a-ce20674464be None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:47.998873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 156/308] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:13:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:48.005330 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af8980-c143-4c79-a47a-55f1bb851152 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:48.052675 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:48.120181 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4af8980-c143-4c79-a47a-55f1bb851152 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:48.155625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074c802a-a3a4-40f4-97ea-c708854768eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:48.156807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c802a-a3a4-40f4-97ea-c708854768eb None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:48.157235 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c802a-a3a4-40f4-97ea-c708854768eb None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:48.157693 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074c802a-a3a4-40f4-97ea-c708854768eb None admin] PATH_INFO: `/v3/users/a525304a661b4698b194e55640479bfd` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:48.159067 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c802a-a3a4-40f4-97ea-c708854768eb None admin] RBAC: Authorizing `identity:get_user(user_id=a525304a661b4698b194e55640479bfd)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:48.161592 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074c802a-a3a4-40f4-97ea-c708854768eb None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:48.163826 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 153/309] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:48 2022] GET /identity/v3/users/a525304a661b4698b194e55640479bfd => generated 293 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:48.169329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfdbee5-6c55-49ba-ad9a-ce20674464be None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:48.290660 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfdbee5-6c55-49ba-ad9a-ce20674464be None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:48.327432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700c224b-df25-499d-a9e0-09039eec2129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:48.328667 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c224b-df25-499d-a9e0-09039eec2129 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:48.329171 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c224b-df25-499d-a9e0-09039eec2129 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:48.329655 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700c224b-df25-499d-a9e0-09039eec2129 None admin] PATH_INFO: `/v3/system/users/a525304a661b4698b194e55640479bfd/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:48.331401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c224b-df25-499d-a9e0-09039eec2129 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a525304a661b4698b194e55640479bfd, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:48.333561 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700c224b-df25-499d-a9e0-09039eec2129 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:48.414183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 157/310] 10.222.0.47 () {58 vars in 1334 bytes} [Thu Jun 30 10:13:48 2022] PUT /identity/v3/system/users/a525304a661b4698b194e55640479bfd/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:13:48.493586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39102b-d01a-4f4c-aca2-2fe0f096538e None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:48.493908 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39102b-d01a-4f4c-aca2-2fe0f096538e None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:48.494157 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39102b-d01a-4f4c-aca2-2fe0f096538e None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:48.496014 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 154/311] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:48.501622 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb1513-9a0c-4f16-a904-40d5ae58281b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:48.501911 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb1513-9a0c-4f16-a904-40d5ae58281b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:48.502199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb1513-9a0c-4f16-a904-40d5ae58281b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:48.510241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:48.522791 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-79cb1513-9a0c-4f16-a904-40d5ae58281b None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:48.524051 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-79cb1513-9a0c-4f16-a904-40d5ae58281b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:48.602316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 158/312] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:48.625995 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39102b-d01a-4f4c-aca2-2fe0f096538e None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:48.731675 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39102b-d01a-4f4c-aca2-2fe0f096538e None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:48.734021 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:48.766766 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4cd1c-34fb-4dd1-9099-44914c5f75ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:48.767587 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4cd1c-34fb-4dd1-9099-44914c5f75ce None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:48.767873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4cd1c-34fb-4dd1-9099-44914c5f75ce None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:48.768150 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4cd1c-34fb-4dd1-9099-44914c5f75ce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:48.779527 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4cd1c-34fb-4dd1-9099-44914c5f75ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:48.787676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-86b4cd1c-34fb-4dd1-9099-44914c5f75ce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 10:13:48.787676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:48.787676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:48.787676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:48.787676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:48.787676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:48.787676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:48.787676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:48.787676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:48.787676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:48.787676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:48.787676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:48.787676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:48.787676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:48.787676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:48.787676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:48.788540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:48.788540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:48.788540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:48.788540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:48.788540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:48.788540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:48.788540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:48.788540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:48.788540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:48.788540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:48.788540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:48.788540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:48.788540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:48.788540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:48.788540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:48.788540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:48.789326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:48.789326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:48.789326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:48.789326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:48.789326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:48.789326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:48.789326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:48.789326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:48.789326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:48.789326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:48.789326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 10:13:48.789326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:13:48.789326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 155/313] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:48.795352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb1513-9a0c-4f16-a904-40d5ae58281b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:48.911520 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb1513-9a0c-4f16-a904-40d5ae58281b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:48.949410 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ef40c-3dd1-40a9-bf65-75db75c710b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:48.950857 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ef40c-3dd1-40a9-bf65-75db75c710b6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:48.951356 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ef40c-3dd1-40a9-bf65-75db75c710b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:48.951815 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116ef40c-3dd1-40a9-bf65-75db75c710b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:48.952814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116ef40c-3dd1-40a9-bf65-75db75c710b6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:48.953311 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116ef40c-3dd1-40a9-bf65-75db75c710b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:48.954948 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116ef40c-3dd1-40a9-bf65-75db75c710b6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:48.959647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:48.968864 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 159/314] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:13:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:48.974873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4cd1c-34fb-4dd1-9099-44914c5f75ce None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:49.092211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4cd1c-34fb-4dd1-9099-44914c5f75ce None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:49.132469 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cea70f-5c9f-43fa-8bc2-4f6d9b8169c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:49.133670 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cea70f-5c9f-43fa-8bc2-4f6d9b8169c2 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:49.134120 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cea70f-5c9f-43fa-8bc2-4f6d9b8169c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:49.136393 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cea70f-5c9f-43fa-8bc2-4f6d9b8169c2 None admin] PATH_INFO: `/v3/users/af6d27a7a2a24f11ad70678a3d01a87e` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:49.136393 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cea70f-5c9f-43fa-8bc2-4f6d9b8169c2 None admin] RBAC: Authorizing `identity:get_user(user_id=af6d27a7a2a24f11ad70678a3d01a87e)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:49.137740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cea70f-5c9f-43fa-8bc2-4f6d9b8169c2 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:49.139994 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 156/315] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:48 2022] GET /identity/v3/users/af6d27a7a2a24f11ad70678a3d01a87e => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:49.145915 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ef40c-3dd1-40a9-bf65-75db75c710b6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:49.257935 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116ef40c-3dd1-40a9-bf65-75db75c710b6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:49.292981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1685d-9a56-4f68-9ce4-2caeb2e9b1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:49.293633 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1685d-9a56-4f68-9ce4-2caeb2e9b1c3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:49.293922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1685d-9a56-4f68-9ce4-2caeb2e9b1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:49.294204 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1685d-9a56-4f68-9ce4-2caeb2e9b1c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:49.367996 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:49.371203 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1685d-9a56-4f68-9ce4-2caeb2e9b1c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:49.372744 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1685d-9a56-4f68-9ce4-2caeb2e9b1c3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:49.500807 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 160/316] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:13:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:13:49.508038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cea70f-5c9f-43fa-8bc2-4f6d9b8169c2 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:49.542357 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:49.621527 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cea70f-5c9f-43fa-8bc2-4f6d9b8169c2 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:49.657824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254a0224-3fe1-4483-bfbd-dcf89892d1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:49.658690 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a0224-3fe1-4483-bfbd-dcf89892d1b7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:49.658975 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a0224-3fe1-4483-bfbd-dcf89892d1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:49.659253 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254a0224-3fe1-4483-bfbd-dcf89892d1b7 None admin] PATH_INFO: `/v3/projects/95abfdd95158427e9f9f0571955df6c4` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:49.661085 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a0224-3fe1-4483-bfbd-dcf89892d1b7 None admin] RBAC: Authorizing `identity:get_project(project_id=95abfdd95158427e9f9f0571955df6c4)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:49.662528 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254a0224-3fe1-4483-bfbd-dcf89892d1b7 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:49.664614 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 157/317] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:49 2022] GET /identity/v3/projects/95abfdd95158427e9f9f0571955df6c4 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:49.670679 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1685d-9a56-4f68-9ce4-2caeb2e9b1c3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:49.788953 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:49.795899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1685d-9a56-4f68-9ce4-2caeb2e9b1c3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:49.833661 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585be7b2-4b8c-4839-8f52-c4e8a1d51bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:49.834686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585be7b2-4b8c-4839-8f52-c4e8a1d51bec None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:49.835172 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585be7b2-4b8c-4839-8f52-c4e8a1d51bec None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:49.835702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585be7b2-4b8c-4839-8f52-c4e8a1d51bec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:49.836736 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585be7b2-4b8c-4839-8f52-c4e8a1d51bec None admin] RBAC: Adding query filter params (role.id=ea0b118209d448f2902b045a6ac12919, scope.project.id=95abfdd95158427e9f9f0571955df6c4, user.id=af6d27a7a2a24f11ad70678a3d01a87e) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:49.837253 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585be7b2-4b8c-4839-8f52-c4e8a1d51bec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:49.838908 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585be7b2-4b8c-4839-8f52-c4e8a1d51bec None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:49.845810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 161/318] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:13:49 2022] GET /identity/v3/role_assignments?user.id=af6d27a7a2a24f11ad70678a3d01a87e&scope.project.id=95abfdd95158427e9f9f0571955df6c4&role.id=ea0b118209d448f2902b045a6ac12919 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:49.935055 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133645d-d422-49cf-a57c-10c2136afee1 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:49.935390 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133645d-d422-49cf-a57c-10c2136afee1 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:49.935651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133645d-d422-49cf-a57c-10c2136afee1 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:49.937347 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 158/319] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:49.943621 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b609442-301f-4ba4-878e-5f12b8f20249 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:49.943910 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b609442-301f-4ba4-878e-5f12b8f20249 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:49.944192 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b609442-301f-4ba4-878e-5f12b8f20249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:49.962269 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5b609442-301f-4ba4-878e-5f12b8f20249 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:49.963515 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5b609442-301f-4ba4-878e-5f12b8f20249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:50.040633 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 162/320] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:50.064222 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133645d-d422-49cf-a57c-10c2136afee1 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:50.139948 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540c658-a853-4873-8470-cce2a9d924e2 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:50.139948 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540c658-a853-4873-8470-cce2a9d924e2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:50.139948 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7540c658-a853-4873-8470-cce2a9d924e2 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:50.139948 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 163/321] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:50.151875 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497e2f6-eeba-4173-b784-c9926f2f62df None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:50.152437 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497e2f6-eeba-4173-b784-c9926f2f62df None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:50.152718 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d497e2f6-eeba-4173-b784-c9926f2f62df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:50.181424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d497e2f6-eeba-4173-b784-c9926f2f62df None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:50.182874 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d497e2f6-eeba-4173-b784-c9926f2f62df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:50.195335 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:50.200321 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133645d-d422-49cf-a57c-10c2136afee1 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:50.239531 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad9564-a7fe-4103-b08e-8d7b83d038bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:50.240394 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad9564-a7fe-4103-b08e-8d7b83d038bc None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:50.240623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad9564-a7fe-4103-b08e-8d7b83d038bc None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:50.240898 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad9564-a7fe-4103-b08e-8d7b83d038bc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:50.242074 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad9564-a7fe-4103-b08e-8d7b83d038bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:50.243542 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ad9564-a7fe-4103-b08e-8d7b83d038bc None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:50.245406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 159/322] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:13:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:50.250856 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad9564-a7fe-4103-b08e-8d7b83d038bc None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:50.274285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 164/323] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:50.308181 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d497e2f6-eeba-4173-b784-c9926f2f62df None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:50.332903 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:50.416164 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad9564-a7fe-4103-b08e-8d7b83d038bc None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:50.423326 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d497e2f6-eeba-4173-b784-c9926f2f62df None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:50.453550 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8cf250-8e9b-4fb5-9965-8f15255e644b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:50.454234 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cf250-8e9b-4fb5-9965-8f15255e644b None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:50.454669 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cf250-8e9b-4fb5-9965-8f15255e644b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:50.454811 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8cf250-8e9b-4fb5-9965-8f15255e644b None admin] PATH_INFO: `/v3/groups` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:50.455417 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8cf250-8e9b-4fb5-9965-8f15255e644b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:50.457835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8cf250-8e9b-4fb5-9965-8f15255e644b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:50.490343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa1d41-f25f-4da3-a899-71a688ed2499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:50.491155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa1d41-f25f-4da3-a899-71a688ed2499 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:50.491479 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa1d41-f25f-4da3-a899-71a688ed2499 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:50.491753 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa1d41-f25f-4da3-a899-71a688ed2499 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:50.502969 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffa1d41-f25f-4da3-a899-71a688ed2499 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:50.511649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-fffa1d41-f25f-4da3-a899-71a688ed2499 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 10:13:50.511649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:50.511649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:50.511649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:50.511649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:50.511649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:50.511649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:50.511649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:50.511649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:50.511649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:50.511649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:50.511649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:50.511649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:50.511649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:50.511649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:50.511649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:50.512767 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:50.512767 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:50.512767 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:50.512767 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:50.512767 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:50.512767 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:50.512767 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:50.512767 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:50.512767 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:50.512767 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:50.512767 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:50.512767 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:50.512767 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:50.512767 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:50.512767 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:50.512767 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:50.513569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:50.513569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:50.513569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:50.513569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:50.513569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:50.513569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:50.513569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:50.513569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:50.513569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:50.513569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:50.513569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 10:13:50.513569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:13:50.513569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 165/324] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:13:50.518752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa1d41-f25f-4da3-a899-71a688ed2499 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:50.544724 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 160/325] 10.222.0.47 () {60 vars in 1134 bytes} [Thu Jun 30 10:13:50 2022] POST /identity/v3/groups => generated 226 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:13:50.621857 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:50.642998 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffa1d41-f25f-4da3-a899-71a688ed2499 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:50.651963 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6785356-0545-4779-88bf-d1eb17f86a28 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:50.652258 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6785356-0545-4779-88bf-d1eb17f86a28 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:50.654126 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6785356-0545-4779-88bf-d1eb17f86a28 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:50.654126 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 161/326] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:13:50.659667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527a124-978d-4b3f-b1f6-b000263f2cd5 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:50.660001 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527a124-978d-4b3f-b1f6-b000263f2cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:50.660297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3527a124-978d-4b3f-b1f6-b000263f2cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:50.678003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-3527a124-978d-4b3f-b1f6-b000263f2cd5 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:50.679221 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-3527a124-978d-4b3f-b1f6-b000263f2cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:50.683302 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7910cd-fc3e-488e-b2ce-5b7558920def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:50.683998 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7910cd-fc3e-488e-b2ce-5b7558920def None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:50.684285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7910cd-fc3e-488e-b2ce-5b7558920def None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:50.684562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7910cd-fc3e-488e-b2ce-5b7558920def None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:50.685338 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7910cd-fc3e-488e-b2ce-5b7558920def None admin] RBAC: Adding query filter params (name=reader) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:50.685653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7910cd-fc3e-488e-b2ce-5b7558920def None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:50.687082 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7910cd-fc3e-488e-b2ce-5b7558920def None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:50.697939 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 166/327] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:13:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:50.703742 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7910cd-fc3e-488e-b2ce-5b7558920def None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:50.709476 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:50.759501 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 162/328] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 10:13:50.784490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527a124-978d-4b3f-b1f6-b000263f2cd5 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:50.821859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7910cd-fc3e-488e-b2ce-5b7558920def None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:50.863873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed312e1-6d3f-4f89-a766-b94a98539b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:50.864764 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed312e1-6d3f-4f89-a766-b94a98539b3a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:50.865645 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed312e1-6d3f-4f89-a766-b94a98539b3a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:50.865645 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed312e1-6d3f-4f89-a766-b94a98539b3a None admin] PATH_INFO: `/v3/users/461011f415a8438dbe71bfc473184cf7` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:50.867035 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed312e1-6d3f-4f89-a766-b94a98539b3a None admin] RBAC: Authorizing `identity:get_user(user_id=461011f415a8438dbe71bfc473184cf7)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:50.868543 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed312e1-6d3f-4f89-a766-b94a98539b3a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:50.870519 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 167/329] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:50 2022] GET /identity/v3/users/461011f415a8438dbe71bfc473184cf7 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:50.876523 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed312e1-6d3f-4f89-a766-b94a98539b3a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:50.898289 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:50.921748 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3527a124-978d-4b3f-b1f6-b000263f2cd5 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:50.973520 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccaab9f-54c2-4d5a-be9c-7b84a14facbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:50.974546 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccaab9f-54c2-4d5a-be9c-7b84a14facbd None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:50.974834 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccaab9f-54c2-4d5a-be9c-7b84a14facbd None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:50.975114 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccaab9f-54c2-4d5a-be9c-7b84a14facbd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:50.990595 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccaab9f-54c2-4d5a-be9c-7b84a14facbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:51.003210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-7ccaab9f-54c2-4d5a-be9c-7b84a14facbd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:51.003210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:51.003210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:51.003210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:51.003210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:51.003210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:51.003210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:51.003210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:51.003210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:51.003210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:51.003210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:51.003210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:51.003210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:51.003210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:51.003210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:51.003210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:51.004785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:51.004785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:51.004785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:51.004785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:51.004785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:51.004785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:51.004785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:51.004785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:51.004785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:51.004785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:51.004785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:51.004785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:51.004785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:51.004785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:51.004785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:51.004785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:51.005605 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:51.005605 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:51.005605 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:51.005605 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:51.005605 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:51.005605 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:51.005605 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:51.005605 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:51.005605 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:51.005605 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:51.005605 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:51.005605 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:13:51.006702 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 163/330] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:13:51.012046 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccaab9f-54c2-4d5a-be9c-7b84a14facbd None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:51.022475 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed312e1-6d3f-4f89-a766-b94a98539b3a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:51.038609 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:51.067293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0408da3-3be4-48aa-bf49-4400659aacf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:51.068334 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0408da3-3be4-48aa-bf49-4400659aacf1 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:51.068984 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0408da3-3be4-48aa-bf49-4400659aacf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:51.069288 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0408da3-3be4-48aa-bf49-4400659aacf1 None admin] PATH_INFO: `/v3/system/users/461011f415a8438dbe71bfc473184cf7/roles/ea0b118209d448f2902b045a6ac12919` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:51.070812 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0408da3-3be4-48aa-bf49-4400659aacf1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=461011f415a8438dbe71bfc473184cf7, role_id=ea0b118209d448f2902b045a6ac12919)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:51.072677 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0408da3-3be4-48aa-bf49-4400659aacf1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:51.126480 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccaab9f-54c2-4d5a-be9c-7b84a14facbd None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:51.142755 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 168/331] 10.222.0.47 () {58 vars in 1334 bytes} [Thu Jun 30 10:13:50 2022] PUT /identity/v3/system/users/461011f415a8438dbe71bfc473184cf7/roles/ea0b118209d448f2902b045a6ac12919 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:13:51.167675 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7f69f-1911-427e-856e-ddfdda18d0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:51.168381 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7f69f-1911-427e-856e-ddfdda18d0ab None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:51.168668 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7f69f-1911-427e-856e-ddfdda18d0ab None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:51.171624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7f69f-1911-427e-856e-ddfdda18d0ab None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:51.171624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7f69f-1911-427e-856e-ddfdda18d0ab None admin] RBAC: Adding query filter params (name=member) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:51.171624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7f69f-1911-427e-856e-ddfdda18d0ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:51.171624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c7f69f-1911-427e-856e-ddfdda18d0ab None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:51.182672 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 164/332] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:13:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:51.188139 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0408da3-3be4-48aa-bf49-4400659aacf1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:51.324331 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:51.350894 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0408da3-3be4-48aa-bf49-4400659aacf1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:51.389117 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a178b83d-7d58-488e-a338-777deb212686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:51.390002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178b83d-7d58-488e-a338-777deb212686 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:51.390290 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178b83d-7d58-488e-a338-777deb212686 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:51.390568 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a178b83d-7d58-488e-a338-777deb212686 None admin] PATH_INFO: `/v3/users/a525304a661b4698b194e55640479bfd` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:51.391839 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a178b83d-7d58-488e-a338-777deb212686 None admin] RBAC: Authorizing `identity:get_user(user_id=a525304a661b4698b194e55640479bfd)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:51.393357 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a178b83d-7d58-488e-a338-777deb212686 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:51.395350 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 169/333] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:51 2022] GET /identity/v3/users/a525304a661b4698b194e55640479bfd => generated 293 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:51.404702 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7f69f-1911-427e-856e-ddfdda18d0ab None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:51.518102 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:51.539624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7f69f-1911-427e-856e-ddfdda18d0ab None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:51.579820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408931e7-8918-4cd4-afe1-d05b2492caa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:51.580493 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408931e7-8918-4cd4-afe1-d05b2492caa3 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:51.580781 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408931e7-8918-4cd4-afe1-d05b2492caa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:51.581060 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408931e7-8918-4cd4-afe1-d05b2492caa3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:51.582450 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408931e7-8918-4cd4-afe1-d05b2492caa3 None admin] RBAC: Adding query filter params (role.id=31034f95ef88402296c753c2c259f717, user.id=a525304a661b4698b194e55640479bfd, scope.system=all) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:51.582799 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408931e7-8918-4cd4-afe1-d05b2492caa3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:51.584326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408931e7-8918-4cd4-afe1-d05b2492caa3 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:51.602915 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 165/334] 10.222.0.47 () {56 vars in 1308 bytes} [Thu Jun 30 10:13:51 2022] GET /identity/v3/role_assignments?user.id=a525304a661b4698b194e55640479bfd&scope.system=all&role.id=31034f95ef88402296c753c2c259f717 => generated 510 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:53.814127 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd77699-a217-4761-882c-39f8b838197e None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:53.815100 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd77699-a217-4761-882c-39f8b838197e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:53.815608 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd77699-a217-4761-882c-39f8b838197e None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:53.817382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 170/335] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:53.822856 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58a995-1bc8-4c55-820c-ff5840167860 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:53.823680 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58a995-1bc8-4c55-820c-ff5840167860 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:53.824175 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a58a995-1bc8-4c55-820c-ff5840167860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:53.848890 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-9a58a995-1bc8-4c55-820c-ff5840167860 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:53.850101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-9a58a995-1bc8-4c55-820c-ff5840167860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:53.942629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 166/336] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:53.944625 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716801ae-6844-4819-a6e4-0dbb4de56e9a None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:53.945125 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716801ae-6844-4819-a6e4-0dbb4de56e9a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:53.945603 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716801ae-6844-4819-a6e4-0dbb4de56e9a None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:53.947438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 171/337] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:53.952899 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1658b-8fbc-493c-a3a8-ccb3089badc5 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:53.953188 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1658b-8fbc-493c-a3a8-ccb3089badc5 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:53.953472 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e1658b-8fbc-493c-a3a8-ccb3089badc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:53.972575 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73465331-9c32-4513-9302-39c23301760c None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:53.973565 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73465331-9c32-4513-9302-39c23301760c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:53.974067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73465331-9c32-4513-9302-39c23301760c None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:53.975951 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 172/338] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:53.977062 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73465331-9c32-4513-9302-39c23301760c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:53.978313 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-e3e1658b-8fbc-493c-a3a8-ccb3089badc5 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:53.979557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-e3e1658b-8fbc-493c-a3a8-ccb3089badc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:54.095543 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:54.103198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 167/339] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:54.104522 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba83a0-75cb-47ec-b4be-6271af023793 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:54.104814 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba83a0-75cb-47ec-b4be-6271af023793 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:54.105097 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba83a0-75cb-47ec-b4be-6271af023793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:54.134118 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-1bba83a0-75cb-47ec-b4be-6271af023793 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:54.135368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-1bba83a0-75cb-47ec-b4be-6271af023793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:54.170822 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73465331-9c32-4513-9302-39c23301760c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:54.239191 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fb8862-1ab1-48ea-ba74-8d1b9ec5140c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:54.240023 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb8862-1ab1-48ea-ba74-8d1b9ec5140c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:54.240304 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb8862-1ab1-48ea-ba74-8d1b9ec5140c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:54.240581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fb8862-1ab1-48ea-ba74-8d1b9ec5140c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:54.261264 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 168/340] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:54.262575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba83a0-75cb-47ec-b4be-6271af023793 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:54.264011 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fb8862-1ab1-48ea-ba74-8d1b9ec5140c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:54.274458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-f9fb8862-1ab1-48ea-ba74-8d1b9ec5140c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:54.274458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:54.274458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:54.274458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:54.274458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:54.274458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:54.274458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:54.274458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:54.274458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:54.274458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:54.274458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:54.274458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:54.274458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:54.274458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:54.274458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:54.274458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:54.275438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:54.275438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:54.275438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:54.275438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:54.275438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:54.275438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:54.275438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:54.275438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:54.275438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:54.275438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:54.275438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:54.275438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:54.275438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:54.275438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:54.275438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:54.275438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:54.276234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:54.276234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:54.276234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:54.276234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:54.276234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:54.276234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:54.276234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:54.276234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:54.276234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:54.276234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:54.276234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:54.276234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:13:54.279382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 173/341] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:54.280826 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f7e59-96d1-4dc4-9397-3bd6e4b3e708 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:54.281312 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f7e59-96d1-4dc4-9397-3bd6e4b3e708 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:54.282132 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699f7e59-96d1-4dc4-9397-3bd6e4b3e708 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:54.283709 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 174/342] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:54.292729 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f7e59-96d1-4dc4-9397-3bd6e4b3e708 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:54.305100 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:54.407365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba83a0-75cb-47ec-b4be-6271af023793 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:54.430231 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699f7e59-96d1-4dc4-9397-3bd6e4b3e708 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:54.440584 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:54.451224 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdd7f83-edad-40a8-98fb-92c198f91ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:54.452112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd7f83-edad-40a8-98fb-92c198f91ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:54.452553 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd7f83-edad-40a8-98fb-92c198f91ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:54.452840 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdd7f83-edad-40a8-98fb-92c198f91ed7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:54.465911 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdd7f83-edad-40a8-98fb-92c198f91ed7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:54.472380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a95883-73cf-41d9-9f41-918e1f15f677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:54.473149 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a95883-73cf-41d9-9f41-918e1f15f677 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:54.473432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a95883-73cf-41d9-9f41-918e1f15f677 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:54.473707 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a95883-73cf-41d9-9f41-918e1f15f677 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:54.474625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-dfdd7f83-edad-40a8-98fb-92c198f91ed7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:54.474625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:54.474625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:54.474625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:54.474625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:54.474625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:54.474625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:54.474625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:54.474625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:54.474625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:54.474625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:54.474625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:54.474625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:54.474625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:54.474625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:54.474625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:54.475490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:54.475490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:54.475490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:54.475490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:54.475490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:54.475490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:54.475490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:54.475490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:54.475490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:54.475490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:54.475490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:54.475490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:54.475490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:54.475490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:54.475490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:54.475490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:54.476270 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:54.476270 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:54.476270 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:54.476270 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:54.476270 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:54.476270 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:54.476270 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:54.476270 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:54.476270 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:54.476270 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:54.476270 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:54.476270 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:13:54.476866 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 169/343] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:54.483296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70283c5f-ea4a-4845-b202-8fd9959d8ba7 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:54.483296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70283c5f-ea4a-4845-b202-8fd9959d8ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:54.483296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70283c5f-ea4a-4845-b202-8fd9959d8ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:54.485029 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a95883-73cf-41d9-9f41-918e1f15f677 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:54.494941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-08a95883-73cf-41d9-9f41-918e1f15f677 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 10:13:54.494941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:54.494941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:54.494941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:54.494941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:54.494941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:54.494941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:54.494941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:54.494941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:54.494941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:54.494941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:54.494941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:54.494941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:54.494941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:54.494941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:54.494941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:54.497045 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:54.497045 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:54.497045 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:54.497045 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:54.497045 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:54.497045 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:54.497045 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:54.497045 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:54.497045 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:54.497045 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:54.497045 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:54.497045 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:54.497045 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:54.497045 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:54.497045 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:54.497045 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:54.499067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:54.499067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:54.499067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:54.499067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:54.499067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:54.499067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:54.499067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:54.499067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:54.499067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:54.499067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:54.499067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 10:13:54.499067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:13:54.499067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 175/344] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jun 30 10:13:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:54.499067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a95883-73cf-41d9-9f41-918e1f15f677 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:54.512352 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-70283c5f-ea4a-4845-b202-8fd9959d8ba7 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:54.514046 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-70283c5f-ea4a-4845-b202-8fd9959d8ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:54.592032 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:54.602868 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 170/345] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:54.604377 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70283c5f-ea4a-4845-b202-8fd9959d8ba7 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:54.641833 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a95883-73cf-41d9-9f41-918e1f15f677 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:54.685472 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416adc27-76c6-4306-b4fa-1936dd280aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:54.686152 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416adc27-76c6-4306-b4fa-1936dd280aa9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:54.686442 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416adc27-76c6-4306-b4fa-1936dd280aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:54.687449 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416adc27-76c6-4306-b4fa-1936dd280aa9 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:54.687572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416adc27-76c6-4306-b4fa-1936dd280aa9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:54.687812 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416adc27-76c6-4306-b4fa-1936dd280aa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:54.689206 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416adc27-76c6-4306-b4fa-1936dd280aa9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:54.698006 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 176/346] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:13:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:54.698847 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416adc27-76c6-4306-b4fa-1936dd280aa9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:54.727994 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70283c5f-ea4a-4845-b202-8fd9959d8ba7 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:54.747533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:54.768710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b1947-fc3d-4502-9a76-6114e90f1609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:54.770451 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b1947-fc3d-4502-9a76-6114e90f1609 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:54.770950 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b1947-fc3d-4502-9a76-6114e90f1609 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:54.771933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b1947-fc3d-4502-9a76-6114e90f1609 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:54.773507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b1947-fc3d-4502-9a76-6114e90f1609 None admin] RBAC: Adding query filter params (name=member) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:54.774288 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b1947-fc3d-4502-9a76-6114e90f1609 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:54.776353 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b1947-fc3d-4502-9a76-6114e90f1609 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:54.788223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 171/347] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:13:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:54.789455 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b1947-fc3d-4502-9a76-6114e90f1609 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:54.817604 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416adc27-76c6-4306-b4fa-1936dd280aa9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:54.850597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:54.871401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f3e255-5c8f-4639-b0d3-b929e3fe7e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:54.872823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3e255-5c8f-4639-b0d3-b929e3fe7e42 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:54.872823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3e255-5c8f-4639-b0d3-b929e3fe7e42 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:54.872823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f3e255-5c8f-4639-b0d3-b929e3fe7e42 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:54.873465 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f3e255-5c8f-4639-b0d3-b929e3fe7e42 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:54.873778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f3e255-5c8f-4639-b0d3-b929e3fe7e42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:54.875173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f3e255-5c8f-4639-b0d3-b929e3fe7e42 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:54.889416 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 177/348] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:13:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:54.890788 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f3e255-5c8f-4639-b0d3-b929e3fe7e42 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:54.924267 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b1947-fc3d-4502-9a76-6114e90f1609 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:54.976080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c0550-9bbe-4066-90f0-292151f78094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:54.976811 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c0550-9bbe-4066-90f0-292151f78094 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:54.977097 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c0550-9bbe-4066-90f0-292151f78094 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:54.977370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c0550-9bbe-4066-90f0-292151f78094 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:54.991901 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c0550-9bbe-4066-90f0-292151f78094 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:54.999812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-ba9c0550-9bbe-4066-90f0-292151f78094 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 10:13:54.999812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:54.999812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:54.999812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:54.999812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:54.999812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:54.999812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:54.999812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:54.999812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:54.999812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:54.999812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:54.999812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:54.999812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:54.999812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:54.999812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:54.999812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:55.000746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:55.000746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:55.000746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:55.000746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:55.000746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:55.000746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:55.000746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:55.000746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:55.000746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:55.000746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:55.000746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:55.000746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:55.000746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:55.000746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:55.000746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:55.000746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:55.001585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:55.001585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:55.001585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:55.001585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:55.001585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:55.001585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:55.001585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:55.001585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:55.001585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:55.001585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:55.001585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 30 10:13:55.001585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:13:55.002684 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 172/349] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:55.004455 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c0550-9bbe-4066-90f0-292151f78094 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:55.017634 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f3e255-5c8f-4639-b0d3-b929e3fe7e42 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:55.057350 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:55.070482 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb8716-9e8b-4462-b706-2da829760d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:55.071148 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb8716-9e8b-4462-b706-2da829760d47 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:55.071544 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb8716-9e8b-4462-b706-2da829760d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:55.071830 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb8716-9e8b-4462-b706-2da829760d47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:55.147013 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c0550-9bbe-4066-90f0-292151f78094 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:55.153672 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:55.175766 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb8716-9e8b-4462-b706-2da829760d47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:55.177904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb8716-9e8b-4462-b706-2da829760d47 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:55.191538 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1072f0a9-a100-41a2-a1cd-6d24d55355ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:55.192296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1072f0a9-a100-41a2-a1cd-6d24d55355ea None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:55.192659 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1072f0a9-a100-41a2-a1cd-6d24d55355ea None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:55.193015 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1072f0a9-a100-41a2-a1cd-6d24d55355ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:55.279108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1072f0a9-a100-41a2-a1cd-6d24d55355ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:55.280608 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1072f0a9-a100-41a2-a1cd-6d24d55355ea None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:55.314614 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 178/350] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:13:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:13:55.317123 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999f7d3-b8e3-420d-b9fa-92ec0b96bffd None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:55.317621 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999f7d3-b8e3-420d-b9fa-92ec0b96bffd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:55.318105 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8999f7d3-b8e3-420d-b9fa-92ec0b96bffd None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:55.319890 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 179/351] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:55.320793 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8999f7d3-b8e3-420d-b9fa-92ec0b96bffd None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:55.331393 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:55.411719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 173/352] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:13:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:13:55.412581 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1072f0a9-a100-41a2-a1cd-6d24d55355ea None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:55.433905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:55.451746 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8999f7d3-b8e3-420d-b9fa-92ec0b96bffd None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:55.504462 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf53231-4f18-47d0-b67e-1354eda43766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:55.504937 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf53231-4f18-47d0-b67e-1354eda43766 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:55.505239 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf53231-4f18-47d0-b67e-1354eda43766 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:55.505492 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf53231-4f18-47d0-b67e-1354eda43766 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:55.560290 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1072f0a9-a100-41a2-a1cd-6d24d55355ea None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:55.591107 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf53231-4f18-47d0-b67e-1354eda43766 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:55.592611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf53231-4f18-47d0-b67e-1354eda43766 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:55.606060 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c3f8e-4df9-4682-abfe-18e017f2c174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:55.606753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c3f8e-4df9-4682-abfe-18e017f2c174 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:55.607042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c3f8e-4df9-4682-abfe-18e017f2c174 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:55.607576 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c3f8e-4df9-4682-abfe-18e017f2c174 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:55.609137 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c3f8e-4df9-4682-abfe-18e017f2c174 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:55.609137 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c3f8e-4df9-4682-abfe-18e017f2c174 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:55.610430 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c3f8e-4df9-4682-abfe-18e017f2c174 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:55.623180 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 174/353] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:13:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:55.623180 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a9be7-bf40-405b-9156-ddcd8ff999ee None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:55.623180 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a9be7-bf40-405b-9156-ddcd8ff999ee None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:55.623180 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426a9be7-bf40-405b-9156-ddcd8ff999ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:55.629432 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:55.642690 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-426a9be7-bf40-405b-9156-ddcd8ff999ee None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:55.644856 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-426a9be7-bf40-405b-9156-ddcd8ff999ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:55.691765 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:55.708471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 180/354] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:13:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:13:55.709411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf53231-4f18-47d0-b67e-1354eda43766 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:55.727733 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 175/355] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:55.728524 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a9be7-bf40-405b-9156-ddcd8ff999ee None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:55.835419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf53231-4f18-47d0-b67e-1354eda43766 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:55.860221 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426a9be7-bf40-405b-9156-ddcd8ff999ee None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:55.879568 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616c8dee-b8e5-44f8-ae87-2f5455c0abe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:55.880502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c8dee-b8e5-44f8-ae87-2f5455c0abe8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:55.880790 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c8dee-b8e5-44f8-ae87-2f5455c0abe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:55.881068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616c8dee-b8e5-44f8-ae87-2f5455c0abe8 None admin] PATH_INFO: `/v3/projects/95abfdd95158427e9f9f0571955df6c4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:55.882507 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c8dee-b8e5-44f8-ae87-2f5455c0abe8 None admin] RBAC: Authorizing `identity:get_project(project_id=95abfdd95158427e9f9f0571955df6c4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:55.885304 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616c8dee-b8e5-44f8-ae87-2f5455c0abe8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:55.886959 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 181/356] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:55 2022] GET /identity/v3/projects/95abfdd95158427e9f9f0571955df6c4 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:55.890451 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f941a-cb5f-480a-af9b-6390b2684955 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:55.890752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f941a-cb5f-480a-af9b-6390b2684955 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:55.891544 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f941a-cb5f-480a-af9b-6390b2684955 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:55.892737 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 182/357] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:55.893249 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f941a-cb5f-480a-af9b-6390b2684955 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:55.904636 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:55.911900 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16b020-ee85-4deb-9fc9-09016656296c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:55.921502 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16b020-ee85-4deb-9fc9-09016656296c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:55.921502 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16b020-ee85-4deb-9fc9-09016656296c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:55.921502 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16b020-ee85-4deb-9fc9-09016656296c None admin] PATH_INFO: `/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:55.921502 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16b020-ee85-4deb-9fc9-09016656296c None admin] RBAC: Authorizing `identity:get_project(project_id=47885b7cd44548ce92d8cfb4b55c8a97)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:55.921502 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16b020-ee85-4deb-9fc9-09016656296c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:55.921502 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 176/358] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:55 2022] GET /identity/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:55.921921 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16b020-ee85-4deb-9fc9-09016656296c None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:55.966624 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:56.030699 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f941a-cb5f-480a-af9b-6390b2684955 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:56.044627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16b020-ee85-4deb-9fc9-09016656296c None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:56.072038 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bfdcf7-e5a8-42fc-b8d0-a7d0c405b8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:56.072922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfdcf7-e5a8-42fc-b8d0-a7d0c405b8bf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:56.073208 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfdcf7-e5a8-42fc-b8d0-a7d0c405b8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:56.073597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bfdcf7-e5a8-42fc-b8d0-a7d0c405b8bf None admin] PATH_INFO: `/v3/users/461011f415a8438dbe71bfc473184cf7` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:56.074879 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bfdcf7-e5a8-42fc-b8d0-a7d0c405b8bf None admin] RBAC: Authorizing `identity:get_user(user_id=461011f415a8438dbe71bfc473184cf7)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:56.076379 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bfdcf7-e5a8-42fc-b8d0-a7d0c405b8bf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:56.078070 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 183/359] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:13:55 2022] GET /identity/v3/users/461011f415a8438dbe71bfc473184cf7 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:56.078830 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bfdcf7-e5a8-42fc-b8d0-a7d0c405b8bf None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:56.089038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae30009-5bbe-44bf-8308-133ac3c5dbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:56.090581 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae30009-5bbe-44bf-8308-133ac3c5dbfd None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:56.090581 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae30009-5bbe-44bf-8308-133ac3c5dbfd None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:56.090581 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae30009-5bbe-44bf-8308-133ac3c5dbfd None admin] PATH_INFO: `/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:56.091646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae30009-5bbe-44bf-8308-133ac3c5dbfd None admin] RBAC: Authorizing `identity:get_project(project_id=47885b7cd44548ce92d8cfb4b55c8a97)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:56.093085 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae30009-5bbe-44bf-8308-133ac3c5dbfd None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:56.094804 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 177/360] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:55 2022] GET /identity/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:56.095630 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae30009-5bbe-44bf-8308-133ac3c5dbfd None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:56.179025 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:56.207134 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bfdcf7-e5a8-42fc-b8d0-a7d0c405b8bf None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:56.214751 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae30009-5bbe-44bf-8308-133ac3c5dbfd None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:56.237308 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:56.250700 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90176595-9b9a-4c19-a2c4-76ea517954cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:56.251500 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90176595-9b9a-4c19-a2c4-76ea517954cf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:56.251987 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90176595-9b9a-4c19-a2c4-76ea517954cf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:56.252134 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90176595-9b9a-4c19-a2c4-76ea517954cf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:56.261412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278809ce-92f4-43db-80cc-0fe0ce52734d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:56.262362 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278809ce-92f4-43db-80cc-0fe0ce52734d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:56.262651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278809ce-92f4-43db-80cc-0fe0ce52734d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:56.262928 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278809ce-92f4-43db-80cc-0fe0ce52734d None admin] PATH_INFO: `/v3/groups/2ba4933eecf34efe9993645362010421` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:56.266818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90176595-9b9a-4c19-a2c4-76ea517954cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:56.272183 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278809ce-92f4-43db-80cc-0fe0ce52734d None admin] RBAC: Authorizing `identity:get_group(group_id=2ba4933eecf34efe9993645362010421)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:56.273476 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278809ce-92f4-43db-80cc-0fe0ce52734d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:56.275179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 178/361] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jun 30 10:13:56 2022] GET /identity/v3/groups/2ba4933eecf34efe9993645362010421 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:56.276395 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87310138-62f3-4f5c-8362-060403f3ae8d None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:56.276682 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87310138-62f3-4f5c-8362-060403f3ae8d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:56.276960 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87310138-62f3-4f5c-8362-060403f3ae8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:56.277637 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-90176595-9b9a-4c19-a2c4-76ea517954cf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 10:13:56.277637 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:56.277637 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:56.277637 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:56.277637 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:56.277637 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:56.277637 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:56.277637 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:56.277637 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:56.277637 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:56.277637 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:56.277637 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:56.277637 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:56.277637 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:56.277637 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:56.277637 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:56.278657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:56.278657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:56.278657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:56.278657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:56.278657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:56.278657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:56.278657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:56.278657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:56.278657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:56.278657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:56.278657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:56.278657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:56.278657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:56.278657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:56.278657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:56.278657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:56.279810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:56.279810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:56.279810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:56.279810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:56.279810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:56.279810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:56.279810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:56.279810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:56.279810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:56.279810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:56.279810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 10:13:56.279810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:13:56.279810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 184/362] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jun 30 10:13:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:56.285217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90176595-9b9a-4c19-a2c4-76ea517954cf None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:56.296978 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-87310138-62f3-4f5c-8362-060403f3ae8d None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:56.298171 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-87310138-62f3-4f5c-8362-060403f3ae8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:56.376040 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 179/363] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:56.377568 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87310138-62f3-4f5c-8362-060403f3ae8d None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:56.404859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90176595-9b9a-4c19-a2c4-76ea517954cf None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:56.447818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fa326-da4c-4e6f-8ed2-2641300b771b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:56.448800 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fa326-da4c-4e6f-8ed2-2641300b771b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:56.449088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fa326-da4c-4e6f-8ed2-2641300b771b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:56.449367 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fa326-da4c-4e6f-8ed2-2641300b771b None admin] PATH_INFO: `/v3/groups/2ba4933eecf34efe9993645362010421` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:56.450525 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:56.450646 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456fa326-da4c-4e6f-8ed2-2641300b771b None admin] RBAC: Authorizing `identity:get_group(group_id=2ba4933eecf34efe9993645362010421)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:56.452066 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456fa326-da4c-4e6f-8ed2-2641300b771b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:56.453698 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 185/364] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jun 30 10:13:56 2022] GET /identity/v3/groups/2ba4933eecf34efe9993645362010421 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:56.454479 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fa326-da4c-4e6f-8ed2-2641300b771b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:56.493331 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87310138-62f3-4f5c-8362-060403f3ae8d None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:56.512943 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:56.528200 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4688a60-0594-4e07-b5e5-34e519b5bed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:56.528864 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4688a60-0594-4e07-b5e5-34e519b5bed8 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:56.529155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4688a60-0594-4e07-b5e5-34e519b5bed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:56.529427 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4688a60-0594-4e07-b5e5-34e519b5bed8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:56.530257 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4688a60-0594-4e07-b5e5-34e519b5bed8 None admin] RBAC: Adding query filter params (role.id=ea0b118209d448f2902b045a6ac12919, user.id=461011f415a8438dbe71bfc473184cf7, scope.system=all) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:56.530570 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4688a60-0594-4e07-b5e5-34e519b5bed8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:56.532044 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4688a60-0594-4e07-b5e5-34e519b5bed8 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:56.543451 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 180/365] 10.222.0.47 () {56 vars in 1308 bytes} [Thu Jun 30 10:13:56 2022] GET /identity/v3/role_assignments?user.id=461011f415a8438dbe71bfc473184cf7&scope.system=all&role.id=ea0b118209d448f2902b045a6ac12919 => generated 510 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:56.551542 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4688a60-0594-4e07-b5e5-34e519b5bed8 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:56.567457 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fa326-da4c-4e6f-8ed2-2641300b771b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:56.607056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771c077-7d0a-4075-9dd3-c7d700e7a7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:56.608087 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771c077-7d0a-4075-9dd3-c7d700e7a7ce None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:56.608370 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771c077-7d0a-4075-9dd3-c7d700e7a7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:56.608645 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6771c077-7d0a-4075-9dd3-c7d700e7a7ce None admin] PATH_INFO: `/v3/groups/2ba4933eecf34efe9993645362010421` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:56.609910 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771c077-7d0a-4075-9dd3-c7d700e7a7ce None admin] RBAC: Authorizing `identity:get_group(group_id=2ba4933eecf34efe9993645362010421)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:56.611566 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6771c077-7d0a-4075-9dd3-c7d700e7a7ce None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:56.618961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 186/366] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jun 30 10:13:56 2022] GET /identity/v3/groups/2ba4933eecf34efe9993645362010421 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:56.618961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771c077-7d0a-4075-9dd3-c7d700e7a7ce None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:56.672543 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4688a60-0594-4e07-b5e5-34e519b5bed8 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:56.695530 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:56.712684 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e6d7a-d2ee-4620-89f6-18ba78aad466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:56.713355 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e6d7a-d2ee-4620-89f6-18ba78aad466 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:56.713639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e6d7a-d2ee-4620-89f6-18ba78aad466 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:56.714533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e6d7a-d2ee-4620-89f6-18ba78aad466 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:56.714821 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e6d7a-d2ee-4620-89f6-18ba78aad466 None admin] RBAC: Adding query filter params (group.id=2ba4933eecf34efe9993645362010421, role.id=31034f95ef88402296c753c2c259f717, scope.project.id=95abfdd95158427e9f9f0571955df6c4) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:56.715136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e6d7a-d2ee-4620-89f6-18ba78aad466 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:56.716619 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e6d7a-d2ee-4620-89f6-18ba78aad466 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:56.724589 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 181/367] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jun 30 10:13:56 2022] GET /identity/v3/role_assignments?group.id=2ba4933eecf34efe9993645362010421&scope.project.id=95abfdd95158427e9f9f0571955df6c4&role.id=31034f95ef88402296c753c2c259f717 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:56.724810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e6d7a-d2ee-4620-89f6-18ba78aad466 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:56.735490 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6771c077-7d0a-4075-9dd3-c7d700e7a7ce None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:56.768983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:56.797917 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8b1fc-1429-4072-9cca-e15ba313a9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:56.798604 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8b1fc-1429-4072-9cca-e15ba313a9ca None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:56.798919 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8b1fc-1429-4072-9cca-e15ba313a9ca None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:56.799166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8b1fc-1429-4072-9cca-e15ba313a9ca None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:56.800870 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b8b1fc-1429-4072-9cca-e15ba313a9ca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:56.801753 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b8b1fc-1429-4072-9cca-e15ba313a9ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:56.805617 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b8b1fc-1429-4072-9cca-e15ba313a9ca None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:56.815008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 187/368] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:13:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:56.816442 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8b1fc-1429-4072-9cca-e15ba313a9ca None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:56.879362 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e6d7a-d2ee-4620-89f6-18ba78aad466 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:56.930771 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06500b-d391-41a6-b112-6f4ab8cde4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:56.931514 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06500b-d391-41a6-b112-6f4ab8cde4b4 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:56.931815 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06500b-d391-41a6-b112-6f4ab8cde4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:56.932069 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e06500b-d391-41a6-b112-6f4ab8cde4b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:56.950743 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e06500b-d391-41a6-b112-6f4ab8cde4b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:56.959179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-8e06500b-d391-41a6-b112-6f4ab8cde4b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:13:56.959179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:56.959179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:56.959179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:56.959179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:56.959179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:56.959179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:56.959179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:56.959179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:56.959179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:56.959179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:56.959179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:56.959179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:56.959179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:56.959179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:56.959179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:56.960168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:56.960168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:56.960168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:56.960168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:56.960168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:56.960168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:56.960168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:56.960168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:56.960168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:56.960168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:56.960168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:56.960168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:56.960168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:56.960168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:56.960168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:56.960168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:56.960949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:56.960949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:56.960949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:56.960949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:56.960949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:56.960949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:56.960949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:56.960949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:56.960949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:56.960949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:56.960949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:13:56.960949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:13:56.960949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 182/369] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:13:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:56.961612 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06500b-d391-41a6-b112-6f4ab8cde4b4 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:56.969020 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8b1fc-1429-4072-9cca-e15ba313a9ca None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:57.016346 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4e147-d848-4917-a5e3-2b4f42d73131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:57.016346 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4e147-d848-4917-a5e3-2b4f42d73131 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:57.016346 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4e147-d848-4917-a5e3-2b4f42d73131 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:57.016722 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e4e147-d848-4917-a5e3-2b4f42d73131 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:57.017649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4e147-d848-4917-a5e3-2b4f42d73131 None admin] RBAC: Adding query filter params (group.id=2ba4933eecf34efe9993645362010421, role.id=31034f95ef88402296c753c2c259f717, scope.project.id=47885b7cd44548ce92d8cfb4b55c8a97) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:57.017966 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4e147-d848-4917-a5e3-2b4f42d73131 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:57.020261 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e4e147-d848-4917-a5e3-2b4f42d73131 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:57.025838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 188/370] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jun 30 10:13:56 2022] GET /identity/v3/role_assignments?group.id=2ba4933eecf34efe9993645362010421&scope.project.id=47885b7cd44548ce92d8cfb4b55c8a97&role.id=31034f95ef88402296c753c2c259f717 => generated 260 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:57.026652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4e147-d848-4917-a5e3-2b4f42d73131 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:57.027194 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:57.102972 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:57.114627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e06500b-d391-41a6-b112-6f4ab8cde4b4 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:57.168723 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4407d6-87b9-4754-a4af-9d28d95333ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:57.169395 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4407d6-87b9-4754-a4af-9d28d95333ed None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:57.169822 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4407d6-87b9-4754-a4af-9d28d95333ed None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:57.170116 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4407d6-87b9-4754-a4af-9d28d95333ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:57.171497 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4407d6-87b9-4754-a4af-9d28d95333ed None admin] RBAC: Adding query filter params (group.id=2ba4933eecf34efe9993645362010421, role.id=284d1573e06d47eba0196afbe92d4d7d, scope.project.id=47885b7cd44548ce92d8cfb4b55c8a97) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:57.171951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4407d6-87b9-4754-a4af-9d28d95333ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:57.173597 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4407d6-87b9-4754-a4af-9d28d95333ed None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:57.182802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 183/371] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jun 30 10:13:56 2022] GET /identity/v3/role_assignments?group.id=2ba4933eecf34efe9993645362010421&scope.project.id=47885b7cd44548ce92d8cfb4b55c8a97&role.id=284d1573e06d47eba0196afbe92d4d7d => generated 260 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:57.183545 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e4e147-d848-4917-a5e3-2b4f42d73131 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:57.183669 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4407d6-87b9-4754-a4af-9d28d95333ed None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:57.233186 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba83752-68e7-478d-bf14-34c05724b50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:57.235052 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba83752-68e7-478d-bf14-34c05724b50c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:57.235612 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba83752-68e7-478d-bf14-34c05724b50c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:57.236107 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba83752-68e7-478d-bf14-34c05724b50c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:57.369593 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba83752-68e7-478d-bf14-34c05724b50c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:57.371101 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba83752-68e7-478d-bf14-34c05724b50c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:57.375645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4407d6-87b9-4754-a4af-9d28d95333ed None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:57.389551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:57.419958 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c591ec-c8a6-40f8-bec1-67a541c13e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:57.420634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c591ec-c8a6-40f8-bec1-67a541c13e5b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:57.420925 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c591ec-c8a6-40f8-bec1-67a541c13e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:57.421195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c591ec-c8a6-40f8-bec1-67a541c13e5b None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:57.421962 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c591ec-c8a6-40f8-bec1-67a541c13e5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:57.422247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c591ec-c8a6-40f8-bec1-67a541c13e5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:57.423734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c591ec-c8a6-40f8-bec1-67a541c13e5b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:57.436992 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:57.438849 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 184/372] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:57.444256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c591ec-c8a6-40f8-bec1-67a541c13e5b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:57.503070 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 189/373] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:13:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:13:57.512002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba83752-68e7-478d-bf14-34c05724b50c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:57.648362 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c591ec-c8a6-40f8-bec1-67a541c13e5b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:57.672189 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba83752-68e7-478d-bf14-34c05724b50c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:57.715009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97356e35-d4e7-48b4-ad41-5be7b6c5febf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:57.716062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97356e35-d4e7-48b4-ad41-5be7b6c5febf None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:57.716349 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97356e35-d4e7-48b4-ad41-5be7b6c5febf None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:57.716631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97356e35-d4e7-48b4-ad41-5be7b6c5febf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:57.720452 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee35e413-ec32-47ee-83fe-2a90ffe722f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:57.721314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35e413-ec32-47ee-83fe-2a90ffe722f6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:57.721595 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35e413-ec32-47ee-83fe-2a90ffe722f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:57.722400 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee35e413-ec32-47ee-83fe-2a90ffe722f6 None admin] PATH_INFO: `/v3/projects/95abfdd95158427e9f9f0571955df6c4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:57.724430 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee35e413-ec32-47ee-83fe-2a90ffe722f6 None admin] RBAC: Authorizing `identity:get_project(project_id=95abfdd95158427e9f9f0571955df6c4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:57.725876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee35e413-ec32-47ee-83fe-2a90ffe722f6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:57.737609 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 190/374] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:57 2022] GET /identity/v3/projects/95abfdd95158427e9f9f0571955df6c4 => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:57.738984 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee35e413-ec32-47ee-83fe-2a90ffe722f6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:57.767633 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:57.799670 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:57.860220 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97356e35-d4e7-48b4-ad41-5be7b6c5febf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:57.861764 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97356e35-d4e7-48b4-ad41-5be7b6c5febf None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:57.903538 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee35e413-ec32-47ee-83fe-2a90ffe722f6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:57.964568 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51de47-ecc5-4a0e-bc2b-f13e95bf7383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:57.965490 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51de47-ecc5-4a0e-bc2b-f13e95bf7383 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:57.965710 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51de47-ecc5-4a0e-bc2b-f13e95bf7383 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:57.965983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab51de47-ecc5-4a0e-bc2b-f13e95bf7383 None admin] PATH_INFO: `/v3/groups/2ba4933eecf34efe9993645362010421` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:57.967252 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51de47-ecc5-4a0e-bc2b-f13e95bf7383 None admin] RBAC: Authorizing `identity:get_group(group_id=2ba4933eecf34efe9993645362010421)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:57.968882 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab51de47-ecc5-4a0e-bc2b-f13e95bf7383 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:57.970685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 191/375] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jun 30 10:13:57 2022] GET /identity/v3/groups/2ba4933eecf34efe9993645362010421 => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:57.979585 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51de47-ecc5-4a0e-bc2b-f13e95bf7383 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:58.046950 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 185/376] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:13:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:13:58.056292 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97356e35-d4e7-48b4-ad41-5be7b6c5febf None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:58.143653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:58.166053 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab51de47-ecc5-4a0e-bc2b-f13e95bf7383 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:58.196966 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:58.229419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722c4da-db0a-4cb5-bd90-590bd47176aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:58.229803 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97356e35-d4e7-48b4-ad41-5be7b6c5febf None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:58.230008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722c4da-db0a-4cb5-bd90-590bd47176aa None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:58.230299 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722c4da-db0a-4cb5-bd90-590bd47176aa None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:58.230582 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2722c4da-db0a-4cb5-bd90-590bd47176aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:58.231457 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722c4da-db0a-4cb5-bd90-590bd47176aa None admin] RBAC: Adding query filter params (group.id=2ba4933eecf34efe9993645362010421, role.id=284d1573e06d47eba0196afbe92d4d7d, scope.project.id=95abfdd95158427e9f9f0571955df6c4) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:58.231774 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722c4da-db0a-4cb5-bd90-590bd47176aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:58.233227 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2722c4da-db0a-4cb5-bd90-590bd47176aa None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:58.244954 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 192/377] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jun 30 10:13:57 2022] GET /identity/v3/role_assignments?group.id=2ba4933eecf34efe9993645362010421&scope.project.id=95abfdd95158427e9f9f0571955df6c4&role.id=284d1573e06d47eba0196afbe92d4d7d => generated 260 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:13:58.296838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a932b-44aa-427e-88bd-a1ecbb4bd054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:58.297756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a932b-44aa-427e-88bd-a1ecbb4bd054 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:58.298044 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a932b-44aa-427e-88bd-a1ecbb4bd054 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:58.298325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034a932b-44aa-427e-88bd-a1ecbb4bd054 None admin] PATH_INFO: `/v3/projects/429c0e046ffb41cbb44280918656fef0` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:58.304258 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a932b-44aa-427e-88bd-a1ecbb4bd054 None admin] RBAC: Authorizing `identity:get_project(project_id=429c0e046ffb41cbb44280918656fef0)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:58.305757 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034a932b-44aa-427e-88bd-a1ecbb4bd054 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:58.307651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 186/378] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:58 2022] GET /identity/v3/projects/429c0e046ffb41cbb44280918656fef0 => generated 344 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:58.313273 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722c4da-db0a-4cb5-bd90-590bd47176aa None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:58.437123 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2722c4da-db0a-4cb5-bd90-590bd47176aa None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:58.478446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd546fef-ea45-4db7-ac3c-33ea3785b1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:58.479329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd546fef-ea45-4db7-ac3c-33ea3785b1ec None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:58.479673 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd546fef-ea45-4db7-ac3c-33ea3785b1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:58.479898 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd546fef-ea45-4db7-ac3c-33ea3785b1ec None admin] PATH_INFO: `/v3/groups/d238919d52d44211be02a5e1688bc3e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:58.495990 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd546fef-ea45-4db7-ac3c-33ea3785b1ec None admin] RBAC: Authorizing `identity:get_group(group_id=d238919d52d44211be02a5e1688bc3e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:58.497485 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd546fef-ea45-4db7-ac3c-33ea3785b1ec None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:58.508770 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 193/379] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jun 30 10:13:58 2022] GET /identity/v3/groups/d238919d52d44211be02a5e1688bc3e0 => generated 229 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:58.510038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a932b-44aa-427e-88bd-a1ecbb4bd054 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:58.655706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034a932b-44aa-427e-88bd-a1ecbb4bd054 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:58.706438 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e1403b-bcfe-4dc8-a645-b58a43524093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:58.707106 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1403b-bcfe-4dc8-a645-b58a43524093 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:58.707418 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1403b-bcfe-4dc8-a645-b58a43524093 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:58.707696 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e1403b-bcfe-4dc8-a645-b58a43524093 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:58.708534 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1403b-bcfe-4dc8-a645-b58a43524093 None admin] RBAC: Adding query filter params (group.id=d238919d52d44211be02a5e1688bc3e0, role.id=e63bda042e104a4a93fba15ffcfe1183, scope.project.id=429c0e046ffb41cbb44280918656fef0) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:58.708845 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1403b-bcfe-4dc8-a645-b58a43524093 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:58.710273 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e1403b-bcfe-4dc8-a645-b58a43524093 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:58.719374 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 187/380] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jun 30 10:13:58 2022] GET /identity/v3/role_assignments?group.id=d238919d52d44211be02a5e1688bc3e0&scope.project.id=429c0e046ffb41cbb44280918656fef0&role.id=e63bda042e104a4a93fba15ffcfe1183 => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:59.044282 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286e3b8-3b30-4740-a961-4a5c3d9eb24d None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:59.044801 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286e3b8-3b30-4740-a961-4a5c3d9eb24d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:59.044898 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1286e3b8-3b30-4740-a961-4a5c3d9eb24d None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:59.046603 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 194/381] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:59.053163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48f5dd-5bb7-4e76-9116-8e5252a2f1a7 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:59.053163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48f5dd-5bb7-4e76-9116-8e5252a2f1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:59.053163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e48f5dd-5bb7-4e76-9116-8e5252a2f1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:59.059670 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:59.072186 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-5e48f5dd-5bb7-4e76-9116-8e5252a2f1a7 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:59.073386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-5e48f5dd-5bb7-4e76-9116-8e5252a2f1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:59.142521 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 188/382] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 10:13:59.165568 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1286e3b8-3b30-4740-a961-4a5c3d9eb24d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:59.177209 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:59.274186 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1286e3b8-3b30-4740-a961-4a5c3d9eb24d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:59.309587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d2ad5-0c66-4bb6-917d-3fd8914041b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:59.310303 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d2ad5-0c66-4bb6-917d-3fd8914041b1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:59.310585 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d2ad5-0c66-4bb6-917d-3fd8914041b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:59.310860 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d2ad5-0c66-4bb6-917d-3fd8914041b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:59.322174 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d2ad5-0c66-4bb6-917d-3fd8914041b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:59.330121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-d04d2ad5-0c66-4bb6-917d-3fd8914041b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:59.330121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:59.330121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:59.330121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:59.330121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:59.330121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:59.330121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:59.330121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:59.330121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:59.330121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:59.330121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:59.330121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:59.330121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:59.330121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:59.330121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:59.330121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:59.331050 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:59.331050 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:59.331050 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:59.331050 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:59.331050 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:59.331050 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:59.331050 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:59.331050 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:59.331050 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:59.331050 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:59.331050 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:59.331050 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:59.331050 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:59.331050 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:59.331050 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:59.331050 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:59.332504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:59.332504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:59.332504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:59.332504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:59.332504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:59.332504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:59.332504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:59.332504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:59.332504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:59.332504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:59.332504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:59.332504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:13:59.332504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 195/383] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:59.338495 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e48f5dd-5bb7-4e76-9116-8e5252a2f1a7 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:59.442261 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e48f5dd-5bb7-4e76-9116-8e5252a2f1a7 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:59.474843 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:59.483646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0eb96d-0889-4b8a-bf5d-bccea276b132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:59.484351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0eb96d-0889-4b8a-bf5d-bccea276b132 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:59.484686 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0eb96d-0889-4b8a-bf5d-bccea276b132 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:59.484963 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0eb96d-0889-4b8a-bf5d-bccea276b132 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:59.485790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0eb96d-0889-4b8a-bf5d-bccea276b132 None admin] RBAC: Adding query filter params (name=member) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:13:59.486103 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0eb96d-0889-4b8a-bf5d-bccea276b132 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:59.487566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0eb96d-0889-4b8a-bf5d-bccea276b132 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:59.504808 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d2ad5-0c66-4bb6-917d-3fd8914041b1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:59.514962 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 189/384] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:13:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:59.520555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac94e0-7f29-4219-a8b8-888ff6819cc3 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:59.520856 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac94e0-7f29-4219-a8b8-888ff6819cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:59.521144 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac94e0-7f29-4219-a8b8-888ff6819cc3 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:59.524956 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 190/385] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:13:59.528225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeded97-527d-47f1-b8e1-17a25eaac1dd None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:59.528512 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeded97-527d-47f1-b8e1-17a25eaac1dd None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:59.528796 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeded97-527d-47f1-b8e1-17a25eaac1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:59.562913 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-3aeded97-527d-47f1-b8e1-17a25eaac1dd None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:59.564063 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-3aeded97-527d-47f1-b8e1-17a25eaac1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:59.611650 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:59.643288 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d2ad5-0c66-4bb6-917d-3fd8914041b1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:59.681795 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2b927-fe90-4a6d-a2d4-4521d9c93124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:59.682979 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2b927-fe90-4a6d-a2d4-4521d9c93124 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:59.683487 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2b927-fe90-4a6d-a2d4-4521d9c93124 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:59.683914 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2b927-fe90-4a6d-a2d4-4521d9c93124 None admin] PATH_INFO: `/v3/groups/2ba4933eecf34efe9993645362010421` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:59.685262 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad2b927-fe90-4a6d-a2d4-4521d9c93124 None admin] RBAC: Authorizing `identity:get_group(group_id=2ba4933eecf34efe9993645362010421)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:59.687438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad2b927-fe90-4a6d-a2d4-4521d9c93124 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:59.689538 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 196/386] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jun 30 10:13:59 2022] GET /identity/v3/groups/2ba4933eecf34efe9993645362010421 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:59.696371 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2b927-fe90-4a6d-a2d4-4521d9c93124 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:59.735343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 191/387] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:13:59.758340 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6eb33-fcc4-4231-8128-df96967ab119 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:59.758980 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6eb33-fcc4-4231-8128-df96967ab119 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:59.759670 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b6eb33-fcc4-4231-8128-df96967ab119 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:59.765428 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 192/388] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:13:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:13:59.766540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6eb33-fcc4-4231-8128-df96967ab119 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:13:59.835742 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2b927-fe90-4a6d-a2d4-4521d9c93124 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:59.877132 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b6eb33-fcc4-4231-8128-df96967ab119 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:13:59.882825 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c6c02c-a9fc-4bc9-b504-8a8fd7908044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:59.883730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6c02c-a9fc-4bc9-b504-8a8fd7908044 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:59.884016 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6c02c-a9fc-4bc9-b504-8a8fd7908044 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:59.884294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c6c02c-a9fc-4bc9-b504-8a8fd7908044 None admin] PATH_INFO: `/v3/projects/95abfdd95158427e9f9f0571955df6c4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:59.885512 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c6c02c-a9fc-4bc9-b504-8a8fd7908044 None admin] RBAC: Authorizing `identity:get_project(project_id=95abfdd95158427e9f9f0571955df6c4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:13:59.887534 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c6c02c-a9fc-4bc9-b504-8a8fd7908044 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:13:59.891409 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 197/389] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:13:59 2022] GET /identity/v3/projects/95abfdd95158427e9f9f0571955df6c4 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:13:59.892812 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573a1f5-0560-4535-9df3-660816096e78 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:59.893290 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573a1f5-0560-4535-9df3-660816096e78 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:59.893844 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7573a1f5-0560-4535-9df3-660816096e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:59.905423 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:59.912684 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:13:59.923075 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7573a1f5-0560-4535-9df3-660816096e78 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:13:59.924936 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab66bb-3308-4050-87d6-f2c7e21973a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:13:59.924936 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab66bb-3308-4050-87d6-f2c7e21973a1 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:13:59.924936 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab66bb-3308-4050-87d6-f2c7e21973a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:13:59.925697 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7573a1f5-0560-4535-9df3-660816096e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:13:59.925791 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ab66bb-3308-4050-87d6-f2c7e21973a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:13:59.936263 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ab66bb-3308-4050-87d6-f2c7e21973a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:13:59.945742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-f1ab66bb-3308-4050-87d6-f2c7e21973a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:59.945742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:13:59.945742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:13:59.945742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:13:59.945742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:13:59.945742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:13:59.945742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:13:59.945742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:13:59.945742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:13:59.945742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:13:59.945742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:13:59.945742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:13:59.945742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:13:59.945742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:13:59.945742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:13:59.945742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:13:59.946648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:13:59.946648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:13:59.946648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:13:59.946648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:13:59.946648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:13:59.946648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:13:59.946648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:13:59.946648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:13:59.946648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:13:59.946648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:13:59.946648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:13:59.946648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:13:59.946648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:13:59.946648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:13:59.946648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:13:59.946648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:13:59.947867 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:13:59.947867 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:13:59.947867 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:13:59.947867 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:13:59.947867 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:13:59.947867 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:13:59.947867 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:13:59.947867 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:13:59.947867 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:13:59.947867 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:13:59.947867 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:13:59.947867 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:13:59.947867 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 193/390] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:13:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:13:59.948536 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab66bb-3308-4050-87d6-f2c7e21973a1 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:00.002092 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 198/391] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:13:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:00.004013 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573a1f5-0560-4535-9df3-660816096e78 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:00.086803 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ab66bb-3308-4050-87d6-f2c7e21973a1 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:00.107426 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7573a1f5-0560-4535-9df3-660816096e78 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:00.124221 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e85aec-0df9-49f4-be1f-a6d6f03c0ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:00.125180 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e85aec-0df9-49f4-be1f-a6d6f03c0ecb None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:00.125400 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e85aec-0df9-49f4-be1f-a6d6f03c0ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:00.125694 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e85aec-0df9-49f4-be1f-a6d6f03c0ecb None admin] PATH_INFO: `/v3/projects/95abfdd95158427e9f9f0571955df6c4/groups/2ba4933eecf34efe9993645362010421/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:00.127907 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e85aec-0df9-49f4-be1f-a6d6f03c0ecb None admin] RBAC: Authorizing `identity:create_grant(project_id=95abfdd95158427e9f9f0571955df6c4, group_id=2ba4933eecf34efe9993645362010421, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:00.130011 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e85aec-0df9-49f4-be1f-a6d6f03c0ecb None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:00.147354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242c66e-6093-46c7-b146-9fdb1f091845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:00.148342 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242c66e-6093-46c7-b146-9fdb1f091845 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:00.148849 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242c66e-6093-46c7-b146-9fdb1f091845 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:00.149337 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c242c66e-6093-46c7-b146-9fdb1f091845 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:00.150297 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c242c66e-6093-46c7-b146-9fdb1f091845 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:00.150806 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c242c66e-6093-46c7-b146-9fdb1f091845 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:00.152412 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c242c66e-6093-46c7-b146-9fdb1f091845 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:00.154101 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:00.161612 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 199/392] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:14:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:00.162626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242c66e-6093-46c7-b146-9fdb1f091845 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:00.235294 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 194/393] 10.222.0.47 () {58 vars in 1442 bytes} [Thu Jun 30 10:13:59 2022] PUT /identity/v3/projects/95abfdd95158427e9f9f0571955df6c4/groups/2ba4933eecf34efe9993645362010421/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:14:00.235294 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e85aec-0df9-49f4-be1f-a6d6f03c0ecb None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:00.320576 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c242c66e-6093-46c7-b146-9fdb1f091845 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:00.334717 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:00.379103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14838b74-1f91-404d-b5bb-8b94f0857400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:00.380780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14838b74-1f91-404d-b5bb-8b94f0857400 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:00.381438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14838b74-1f91-404d-b5bb-8b94f0857400 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:00.381904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14838b74-1f91-404d-b5bb-8b94f0857400 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:00.396002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14838b74-1f91-404d-b5bb-8b94f0857400 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:00.405352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-14838b74-1f91-404d-b5bb-8b94f0857400 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 10:14:00.405352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:00.405352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:00.405352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:00.405352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:00.405352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:00.405352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:00.405352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:00.405352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:00.405352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:00.405352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:00.405352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:00.405352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:00.405352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:00.405352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:00.405352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:00.406314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:00.406314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:00.406314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:00.406314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:00.406314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:00.406314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:00.406314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:00.406314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:00.406314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:00.406314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:00.406314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:00.406314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:00.406314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:00.406314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:00.406314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:00.406314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:00.407126 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:00.407126 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:00.407126 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:00.407126 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:00.407126 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:00.407126 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:00.407126 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:00.407126 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:00.407126 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:00.407126 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:00.407126 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 10:14:00.407126 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:00.411124 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 200/394] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jun 30 10:14:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:00.430353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14838b74-1f91-404d-b5bb-8b94f0857400 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:00.455469 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e85aec-0df9-49f4-be1f-a6d6f03c0ecb None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:00.497576 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684bc63-3937-423f-b74a-432cf04fa96d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:00.498449 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684bc63-3937-423f-b74a-432cf04fa96d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:00.498730 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684bc63-3937-423f-b74a-432cf04fa96d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:00.499006 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1684bc63-3937-423f-b74a-432cf04fa96d None admin] PATH_INFO: `/v3/groups/2ba4933eecf34efe9993645362010421` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:00.500408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1684bc63-3937-423f-b74a-432cf04fa96d None admin] RBAC: Authorizing `identity:get_group(group_id=2ba4933eecf34efe9993645362010421)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:00.502072 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1684bc63-3937-423f-b74a-432cf04fa96d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:00.504959 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 195/395] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jun 30 10:14:00 2022] GET /identity/v3/groups/2ba4933eecf34efe9993645362010421 => generated 226 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:00.512970 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684bc63-3937-423f-b74a-432cf04fa96d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:00.521930 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:00.610619 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14838b74-1f91-404d-b5bb-8b94f0857400 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:00.688317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1684bc63-3937-423f-b74a-432cf04fa96d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:00.703988 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:00.712002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b58fe8-b459-40ff-9043-67a3e13696c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:00.712761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b58fe8-b459-40ff-9043-67a3e13696c5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:00.713054 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b58fe8-b459-40ff-9043-67a3e13696c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:00.713334 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b58fe8-b459-40ff-9043-67a3e13696c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:00.714363 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b58fe8-b459-40ff-9043-67a3e13696c5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:00.714484 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b58fe8-b459-40ff-9043-67a3e13696c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:00.715838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b58fe8-b459-40ff-9043-67a3e13696c5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:00.730443 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031ec10-c6c6-4a8c-9834-0b04402b0312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:00.731775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031ec10-c6c6-4a8c-9834-0b04402b0312 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:00.732550 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031ec10-c6c6-4a8c-9834-0b04402b0312 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:00.733056 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2031ec10-c6c6-4a8c-9834-0b04402b0312 None admin] PATH_INFO: `/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:00.734481 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2031ec10-c6c6-4a8c-9834-0b04402b0312 None admin] RBAC: Authorizing `identity:get_project(project_id=47885b7cd44548ce92d8cfb4b55c8a97)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:00.734689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 201/396] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:00.736485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2031ec10-c6c6-4a8c-9834-0b04402b0312 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:00.739356 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 196/397] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:14:00 2022] GET /identity/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97 => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:00.745173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b58fe8-b459-40ff-9043-67a3e13696c5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:00.745319 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031ec10-c6c6-4a8c-9834-0b04402b0312 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:00.861974 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2031ec10-c6c6-4a8c-9834-0b04402b0312 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:00.888068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b58fe8-b459-40ff-9043-67a3e13696c5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:00.899804 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449467a2-3867-492a-992c-d0db6aa5abe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:00.900827 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449467a2-3867-492a-992c-d0db6aa5abe7 None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:00.901219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449467a2-3867-492a-992c-d0db6aa5abe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:00.901600 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449467a2-3867-492a-992c-d0db6aa5abe7 None admin] PATH_INFO: `/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97/groups/2ba4933eecf34efe9993645362010421/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:00.903743 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449467a2-3867-492a-992c-d0db6aa5abe7 None admin] RBAC: Authorizing `identity:create_grant(project_id=47885b7cd44548ce92d8cfb4b55c8a97, group_id=2ba4933eecf34efe9993645362010421, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:00.905283 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449467a2-3867-492a-992c-d0db6aa5abe7 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:00.911996 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:00.929589 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af17389-1fc5-4a65-98fe-bf5105c0a0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:00.930810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af17389-1fc5-4a65-98fe-bf5105c0a0ad None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:00.931327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af17389-1fc5-4a65-98fe-bf5105c0a0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:00.931775 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af17389-1fc5-4a65-98fe-bf5105c0a0ad None admin] PATH_INFO: `/v3/groups/2ba4933eecf34efe9993645362010421` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:00.933149 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af17389-1fc5-4a65-98fe-bf5105c0a0ad None admin] RBAC: Authorizing `identity:get_group(group_id=2ba4933eecf34efe9993645362010421)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:00.934826 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af17389-1fc5-4a65-98fe-bf5105c0a0ad None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:00.942451 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 202/398] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jun 30 10:14:00 2022] GET /identity/v3/groups/2ba4933eecf34efe9993645362010421 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:00.942451 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a81b3-097a-4fb2-8f83-f3655a4ba15d None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:00.942451 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a81b3-097a-4fb2-8f83-f3655a4ba15d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:00.942451 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371a81b3-097a-4fb2-8f83-f3655a4ba15d None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:00.942451 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 203/399] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:00.943154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a81b3-097a-4fb2-8f83-f3655a4ba15d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:01.003925 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 197/400] 10.222.0.47 () {58 vars in 1442 bytes} [Thu Jun 30 10:14:00 2022] PUT /identity/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97/groups/2ba4933eecf34efe9993645362010421/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:14:01.006173 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933da3f-ed1e-4c94-9990-aa504e054d13 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:01.006476 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933da3f-ed1e-4c94-9990-aa504e054d13 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:01.006770 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a933da3f-ed1e-4c94-9990-aa504e054d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:01.045925 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-a933da3f-ed1e-4c94-9990-aa504e054d13 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:01.050630 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-a933da3f-ed1e-4c94-9990-aa504e054d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:01.100202 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:01.140456 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371a81b3-097a-4fb2-8f83-f3655a4ba15d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:01.177121 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 198/401] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:01.205883 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a933da3f-ed1e-4c94-9990-aa504e054d13 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:01.212117 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef6f6d-07d5-496d-97ff-5dccdc4ea1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:01.213000 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef6f6d-07d5-496d-97ff-5dccdc4ea1cc None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:01.213796 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef6f6d-07d5-496d-97ff-5dccdc4ea1cc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:01.213796 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ef6f6d-07d5-496d-97ff-5dccdc4ea1cc None admin] PATH_INFO: `/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:01.215125 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef6f6d-07d5-496d-97ff-5dccdc4ea1cc None admin] RBAC: Authorizing `identity:get_project(project_id=47885b7cd44548ce92d8cfb4b55c8a97)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:01.216687 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ef6f6d-07d5-496d-97ff-5dccdc4ea1cc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:01.219621 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 204/402] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:14:00 2022] GET /identity/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97 => generated 298 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:14:01.226885 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef6f6d-07d5-496d-97ff-5dccdc4ea1cc None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:01.300660 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:01.349361 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a933da3f-ed1e-4c94-9990-aa504e054d13 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:01.355517 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ef6f6d-07d5-496d-97ff-5dccdc4ea1cc None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:01.385797 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4fd441-90ba-48de-97aa-593be2f3ac21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:01.386753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fd441-90ba-48de-97aa-593be2f3ac21 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:01.387098 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fd441-90ba-48de-97aa-593be2f3ac21 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:01.387477 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4fd441-90ba-48de-97aa-593be2f3ac21 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:01.394504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9643fd-46bb-4783-9a59-1dcd5c818084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:01.395447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9643fd-46bb-4783-9a59-1dcd5c818084 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:01.395734 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9643fd-46bb-4783-9a59-1dcd5c818084 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:01.396013 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9643fd-46bb-4783-9a59-1dcd5c818084 None admin] PATH_INFO: `/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97/groups/2ba4933eecf34efe9993645362010421/roles/284d1573e06d47eba0196afbe92d4d7d` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:01.399885 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9643fd-46bb-4783-9a59-1dcd5c818084 None admin] RBAC: Authorizing `identity:create_grant(project_id=47885b7cd44548ce92d8cfb4b55c8a97, group_id=2ba4933eecf34efe9993645362010421, role_id=284d1573e06d47eba0196afbe92d4d7d)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:01.401236 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4fd441-90ba-48de-97aa-593be2f3ac21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:01.402378 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9643fd-46bb-4783-9a59-1dcd5c818084 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:01.411297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-4f4fd441-90ba-48de-97aa-593be2f3ac21 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 10:14:01.411297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:01.411297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:01.411297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:01.411297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:01.411297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:01.411297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:01.411297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:01.411297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:01.411297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:01.411297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:01.411297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:01.411297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:01.411297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:01.411297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:01.411297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:01.420368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:01.420368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:01.420368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:01.420368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:01.420368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:01.420368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:01.420368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:01.420368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:01.420368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:01.420368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:01.420368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:01.420368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:01.420368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:01.420368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:01.420368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:01.420368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:01.421237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:01.421237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:01.421237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:01.421237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:01.421237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:01.421237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:01.421237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:01.421237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:01.421237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:01.421237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:01.421237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 10:14:01.421237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:01.421237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 199/403] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jun 30 10:14:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:14:01.421237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e142d-97fe-43bf-b7b4-d2984607e2e6 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:01.421973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e142d-97fe-43bf-b7b4-d2984607e2e6 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:01.421973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6e142d-97fe-43bf-b7b4-d2984607e2e6 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:01.421973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 200/404] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:01.421973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e142d-97fe-43bf-b7b4-d2984607e2e6 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:01.454365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:01.518409 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 205/405] 10.222.0.47 () {58 vars in 1442 bytes} [Thu Jun 30 10:14:01 2022] PUT /identity/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97/groups/2ba4933eecf34efe9993645362010421/roles/284d1573e06d47eba0196afbe92d4d7d => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:14:01.526125 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaa2f7-1d84-4734-b6f5-f52fe7bf3e81 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:01.526125 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaa2f7-1d84-4734-b6f5-f52fe7bf3e81 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:01.526125 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbaa2f7-1d84-4734-b6f5-f52fe7bf3e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:01.541574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6e142d-97fe-43bf-b7b4-d2984607e2e6 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:01.547798 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5bbaa2f7-1d84-4734-b6f5-f52fe7bf3e81 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:01.549298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5bbaa2f7-1d84-4734-b6f5-f52fe7bf3e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:01.625152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc0b93-4091-4486-bfca-eb447b316187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:01.625965 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc0b93-4091-4486-bfca-eb447b316187 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:01.626335 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc0b93-4091-4486-bfca-eb447b316187 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:01.626926 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfc0b93-4091-4486-bfca-eb447b316187 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:01.627533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc0b93-4091-4486-bfca-eb447b316187 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:01.627952 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc0b93-4091-4486-bfca-eb447b316187 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:01.629428 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfc0b93-4091-4486-bfca-eb447b316187 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:01.643461 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 206/406] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:01.654127 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 201/407] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:01.660803 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbaa2f7-1d84-4734-b6f5-f52fe7bf3e81 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:01.673969 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc0b93-4091-4486-bfca-eb447b316187 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:01.756336 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:01.828881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbaa2f7-1d84-4734-b6f5-f52fe7bf3e81 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:01.834434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:01.847878 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfc0b93-4091-4486-bfca-eb447b316187 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:01.887842 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8040bb09-9458-4c7c-8d0e-47087cde08d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:01.888809 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040bb09-9458-4c7c-8d0e-47087cde08d4 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:01.889199 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040bb09-9458-4c7c-8d0e-47087cde08d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:01.889683 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8040bb09-9458-4c7c-8d0e-47087cde08d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:01.894873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15255b77-f0cc-43f1-8211-2e95c99503ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:01.895948 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15255b77-f0cc-43f1-8211-2e95c99503ec None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:01.896369 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15255b77-f0cc-43f1-8211-2e95c99503ec None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:01.897034 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15255b77-f0cc-43f1-8211-2e95c99503ec None admin] PATH_INFO: `/v3/groups/2ba4933eecf34efe9993645362010421` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:01.901419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8040bb09-9458-4c7c-8d0e-47087cde08d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:01.902215 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15255b77-f0cc-43f1-8211-2e95c99503ec None admin] RBAC: Authorizing `identity:get_group(group_id=2ba4933eecf34efe9993645362010421)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:01.903921 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15255b77-f0cc-43f1-8211-2e95c99503ec None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:01.905699 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 207/408] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jun 30 10:14:01 2022] GET /identity/v3/groups/2ba4933eecf34efe9993645362010421 => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:01.909987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-8040bb09-9458-4c7c-8d0e-47087cde08d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:14:01.909987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:01.909987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:01.909987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:01.909987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:01.909987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:01.909987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:01.909987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:01.909987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:01.909987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:01.909987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:01.909987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:01.909987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:01.909987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:01.909987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:01.909987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:01.911863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:01.911863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:01.911863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:01.911863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:01.911863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:01.911863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:01.911863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:01.911863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:01.911863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:01.911863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:01.911863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:01.911863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:01.911863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:01.911863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:01.911863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:01.911863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:01.912835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:01.912835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:01.912835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:01.912835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:01.912835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:01.912835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:01.912835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:01.912835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:01.912835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:01.912835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:01.912835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:14:01.912835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:01.914593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 202/409] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:14:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:01.919717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15255b77-f0cc-43f1-8211-2e95c99503ec None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:01.920084 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8040bb09-9458-4c7c-8d0e-47087cde08d4 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:02.062738 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8040bb09-9458-4c7c-8d0e-47087cde08d4 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:02.069398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15255b77-f0cc-43f1-8211-2e95c99503ec None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:02.092270 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:02.103098 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a468a-c6ff-420e-9459-f278d8b10aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:02.104043 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a468a-c6ff-420e-9459-f278d8b10aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:02.104537 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a468a-c6ff-420e-9459-f278d8b10aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:02.105008 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4a468a-c6ff-420e-9459-f278d8b10aa6 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:02.105951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a468a-c6ff-420e-9459-f278d8b10aa6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:02.106467 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a468a-c6ff-420e-9459-f278d8b10aa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:02.108049 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4a468a-c6ff-420e-9459-f278d8b10aa6 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:02.118046 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 203/410] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:14:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:02.124443 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a468a-c6ff-420e-9459-f278d8b10aa6 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:02.136013 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e20242d-b43a-4115-833a-0905cd05422b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:02.136897 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20242d-b43a-4115-833a-0905cd05422b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:02.137181 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20242d-b43a-4115-833a-0905cd05422b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:02.137456 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e20242d-b43a-4115-833a-0905cd05422b None admin] PATH_INFO: `/v3/projects/95abfdd95158427e9f9f0571955df6c4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:02.138719 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e20242d-b43a-4115-833a-0905cd05422b None admin] RBAC: Authorizing `identity:get_project(project_id=95abfdd95158427e9f9f0571955df6c4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:02.140203 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e20242d-b43a-4115-833a-0905cd05422b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:02.142201 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 208/411] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:14:01 2022] GET /identity/v3/projects/95abfdd95158427e9f9f0571955df6c4 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:02.150661 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e20242d-b43a-4115-833a-0905cd05422b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:02.152717 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:02.250375 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4a468a-c6ff-420e-9459-f278d8b10aa6 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:02.290150 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e20242d-b43a-4115-833a-0905cd05422b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:02.296778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c2341-f598-4782-9e42-dc6631ffa399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:02.297970 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c2341-f598-4782-9e42-dc6631ffa399 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:02.298448 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c2341-f598-4782-9e42-dc6631ffa399 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:02.298915 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c2341-f598-4782-9e42-dc6631ffa399 None admin] PATH_INFO: `/v3/groups/d238919d52d44211be02a5e1688bc3e0` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:02.301965 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c2341-f598-4782-9e42-dc6631ffa399 None admin] RBAC: Authorizing `identity:get_group(group_id=d238919d52d44211be02a5e1688bc3e0)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:02.303842 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c2341-f598-4782-9e42-dc6631ffa399 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:02.306021 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 204/412] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jun 30 10:14:02 2022] GET /identity/v3/groups/d238919d52d44211be02a5e1688bc3e0 => generated 229 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:02.311807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c2341-f598-4782-9e42-dc6631ffa399 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:02.340653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e0fc5-ae3d-43cb-aebf-0de6bedbbebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:02.341454 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e0fc5-ae3d-43cb-aebf-0de6bedbbebf None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:02.341735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e0fc5-ae3d-43cb-aebf-0de6bedbbebf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:02.342010 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e0fc5-ae3d-43cb-aebf-0de6bedbbebf None admin] PATH_INFO: `/v3/projects/95abfdd95158427e9f9f0571955df6c4/groups/2ba4933eecf34efe9993645362010421/roles/284d1573e06d47eba0196afbe92d4d7d` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:02.346618 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e0fc5-ae3d-43cb-aebf-0de6bedbbebf None admin] RBAC: Authorizing `identity:create_grant(project_id=95abfdd95158427e9f9f0571955df6c4, group_id=2ba4933eecf34efe9993645362010421, role_id=284d1573e06d47eba0196afbe92d4d7d)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:02.347943 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e0fc5-ae3d-43cb-aebf-0de6bedbbebf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:02.408893 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:02.430499 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c2341-f598-4782-9e42-dc6631ffa399 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:02.439170 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 209/413] 10.222.0.47 () {58 vars in 1442 bytes} [Thu Jun 30 10:14:02 2022] PUT /identity/v3/projects/95abfdd95158427e9f9f0571955df6c4/groups/2ba4933eecf34efe9993645362010421/roles/284d1573e06d47eba0196afbe92d4d7d => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:14:02.473609 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147477fa-bff2-4737-b062-9466d7fc29b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:02.474594 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147477fa-bff2-4737-b062-9466d7fc29b1 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:02.474875 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147477fa-bff2-4737-b062-9466d7fc29b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:02.475148 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147477fa-bff2-4737-b062-9466d7fc29b1 None admin] PATH_INFO: `/v3/projects/429c0e046ffb41cbb44280918656fef0` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:02.476375 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147477fa-bff2-4737-b062-9466d7fc29b1 None admin] RBAC: Authorizing `identity:get_project(project_id=429c0e046ffb41cbb44280918656fef0)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:02.477938 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147477fa-bff2-4737-b062-9466d7fc29b1 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:02.480083 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 205/414] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:14:02 2022] GET /identity/v3/projects/429c0e046ffb41cbb44280918656fef0 => generated 344 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:02.492974 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e0fc5-ae3d-43cb-aebf-0de6bedbbebf None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:02.548024 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:02.630916 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e0fc5-ae3d-43cb-aebf-0de6bedbbebf None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:02.642482 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04263c-0734-483e-8cbe-7b0877ef3bd1 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:02.642482 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04263c-0734-483e-8cbe-7b0877ef3bd1 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:02.642482 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce04263c-0734-483e-8cbe-7b0877ef3bd1 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:02.642482 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 206/415] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:02.649729 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002a12d-2d04-4f45-a738-a01bd727f0ce None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:02.650040 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002a12d-2d04-4f45-a738-a01bd727f0ce None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:02.650302 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5002a12d-2d04-4f45-a738-a01bd727f0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:02.675793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-5002a12d-2d04-4f45-a738-a01bd727f0ce None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:02.676371 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-5002a12d-2d04-4f45-a738-a01bd727f0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:02.679632 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c157e-212f-4aaa-bc81-49de0539a01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:02.680873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c157e-212f-4aaa-bc81-49de0539a01b None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:02.681160 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c157e-212f-4aaa-bc81-49de0539a01b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:02.681502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8c157e-212f-4aaa-bc81-49de0539a01b None admin] PATH_INFO: `/v3/projects/429c0e046ffb41cbb44280918656fef0/groups/d238919d52d44211be02a5e1688bc3e0/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:02.683530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c157e-212f-4aaa-bc81-49de0539a01b None admin] RBAC: Authorizing `identity:create_grant(project_id=429c0e046ffb41cbb44280918656fef0, group_id=d238919d52d44211be02a5e1688bc3e0, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:02.684986 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8c157e-212f-4aaa-bc81-49de0539a01b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:02.759047 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 207/416] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:02.769055 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 210/417] 10.222.0.47 () {58 vars in 1442 bytes} [Thu Jun 30 10:14:02 2022] PUT /identity/v3/projects/429c0e046ffb41cbb44280918656fef0/groups/d238919d52d44211be02a5e1688bc3e0/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:14:02.788559 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002a12d-2d04-4f45-a738-a01bd727f0ce None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:02.926291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:02.948486 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5002a12d-2d04-4f45-a738-a01bd727f0ce None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:02.991340 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe81f4-b8c0-457e-ba8b-bc83c3df2875 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:02.992161 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe81f4-b8c0-457e-ba8b-bc83c3df2875 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:02.992447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe81f4-b8c0-457e-ba8b-bc83c3df2875 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:02.992727 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfe81f4-b8c0-457e-ba8b-bc83c3df2875 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:03.005850 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfe81f4-b8c0-457e-ba8b-bc83c3df2875 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:03.013706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-7cfe81f4-b8c0-457e-ba8b-bc83c3df2875 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:14:03.013706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:03.013706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:03.013706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:03.013706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:03.013706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:03.013706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:03.013706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:03.013706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:03.013706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:03.013706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:03.013706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:03.013706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:03.013706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:03.013706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:03.013706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:03.014704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:03.014704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:03.014704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:03.014704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:03.014704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:03.014704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:03.014704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:03.014704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:03.014704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:03.014704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:03.014704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:03.014704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:03.014704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:03.014704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:03.014704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:03.014704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:03.015566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:03.015566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:03.015566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:03.015566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:03.015566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:03.015566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:03.015566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:03.015566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:03.015566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:03.015566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:03.015566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:14:03.015566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:03.015566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 208/418] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:14:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:03.021964 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c157e-212f-4aaa-bc81-49de0539a01b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:03.134092 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:03.144573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8c157e-212f-4aaa-bc81-49de0539a01b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:03.188772 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6bb879-f0cb-4329-bac7-8ddbf1de4181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:03.189384 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6bb879-f0cb-4329-bac7-8ddbf1de4181 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:03.189670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6bb879-f0cb-4329-bac7-8ddbf1de4181 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:03.189950 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6bb879-f0cb-4329-bac7-8ddbf1de4181 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:03.191595 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6bb879-f0cb-4329-bac7-8ddbf1de4181 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:03.191914 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6bb879-f0cb-4329-bac7-8ddbf1de4181 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:03.193360 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6bb879-f0cb-4329-bac7-8ddbf1de4181 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:03.208681 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 211/419] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:14:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:03.213400 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe81f4-b8c0-457e-ba8b-bc83c3df2875 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:03.347189 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfe81f4-b8c0-457e-ba8b-bc83c3df2875 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:03.392400 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9920b-bfe2-452e-8c4f-beef60cbd9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:03.393061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9920b-bfe2-452e-8c4f-beef60cbd9ee None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:03.393450 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9920b-bfe2-452e-8c4f-beef60cbd9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:03.394414 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de9920b-bfe2-452e-8c4f-beef60cbd9ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:03.410001 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:03.483456 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de9920b-bfe2-452e-8c4f-beef60cbd9ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:03.484840 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de9920b-bfe2-452e-8c4f-beef60cbd9ee None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:03.553012 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00499611-418d-4395-b3e3-aab1c5c0f2e1 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:03.553337 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00499611-418d-4395-b3e3-aab1c5c0f2e1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:03.553574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00499611-418d-4395-b3e3-aab1c5c0f2e1 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:03.555191 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 212/420] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:14:03.560002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237d7c7-3117-4345-a3d8-e428421cee3a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:03.560504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237d7c7-3117-4345-a3d8-e428421cee3a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:03.561043 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237d7c7-3117-4345-a3d8-e428421cee3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:03.604257 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1237d7c7-3117-4345-a3d8-e428421cee3a None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:03.605806 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1237d7c7-3117-4345-a3d8-e428421cee3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:03.606006 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 209/421] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:14:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:14:03.613763 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9920b-bfe2-452e-8c4f-beef60cbd9ee None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:03.668306 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:03.711897 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 213/422] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:03.745736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de9920b-bfe2-452e-8c4f-beef60cbd9ee None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:03.795385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237d7c7-3117-4345-a3d8-e428421cee3a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:03.802229 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad99c58-7d74-4061-bdb8-9169d8aeb920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:03.803449 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad99c58-7d74-4061-bdb8-9169d8aeb920 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:03.803945 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad99c58-7d74-4061-bdb8-9169d8aeb920 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:03.804879 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad99c58-7d74-4061-bdb8-9169d8aeb920 None admin] PATH_INFO: `/v3/projects/95abfdd95158427e9f9f0571955df6c4` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:03.806367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad99c58-7d74-4061-bdb8-9169d8aeb920 None admin] RBAC: Authorizing `identity:get_project(project_id=95abfdd95158427e9f9f0571955df6c4)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:03.808076 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad99c58-7d74-4061-bdb8-9169d8aeb920 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:03.810692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 210/423] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:14:03 2022] GET /identity/v3/projects/95abfdd95158427e9f9f0571955df6c4 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:03.816244 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad99c58-7d74-4061-bdb8-9169d8aeb920 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:03.955550 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:03.966096 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237d7c7-3117-4345-a3d8-e428421cee3a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:04.012194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea87032-62e4-4233-852e-bb7d5fe42d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:04.013316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea87032-62e4-4233-852e-bb7d5fe42d5f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:04.013801 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea87032-62e4-4233-852e-bb7d5fe42d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:04.014266 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea87032-62e4-4233-852e-bb7d5fe42d5f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:04.039254 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea87032-62e4-4233-852e-bb7d5fe42d5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:04.048305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-7ea87032-62e4-4233-852e-bb7d5fe42d5f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:14:04.048305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:04.048305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:04.048305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:04.048305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:04.048305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:04.048305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:04.048305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:04.048305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:04.048305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:04.048305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:04.048305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:04.048305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:04.048305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:04.048305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:04.048305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:04.049195 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:04.049195 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:04.049195 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:04.049195 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:04.049195 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:04.049195 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:04.049195 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:04.049195 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:04.049195 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:04.049195 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:04.049195 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:04.049195 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:04.049195 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:04.049195 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:04.049195 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:04.049195 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:04.050572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:04.050572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:04.050572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:04.050572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:04.050572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:04.050572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:04.050572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:04.050572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:04.050572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:04.050572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:04.050572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 30 10:14:04.050572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:04.053134 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 214/424] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:14:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:14:04.059179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad99c58-7d74-4061-bdb8-9169d8aeb920 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:04.062190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea87032-62e4-4233-852e-bb7d5fe42d5f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:04.071090 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:04.168883 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc3749-9548-48e8-9e3d-3bbbaa7dc9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:04.170074 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc3749-9548-48e8-9e3d-3bbbaa7dc9e8 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:04.170520 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc3749-9548-48e8-9e3d-3bbbaa7dc9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:04.170958 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc3749-9548-48e8-9e3d-3bbbaa7dc9e8 None admin] PATH_INFO: `/v3/groups/2ba4933eecf34efe9993645362010421` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:04.172413 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc3749-9548-48e8-9e3d-3bbbaa7dc9e8 None admin] RBAC: Authorizing `identity:get_group(group_id=2ba4933eecf34efe9993645362010421)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:04.174129 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc3749-9548-48e8-9e3d-3bbbaa7dc9e8 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:04.177003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 211/425] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jun 30 10:14:03 2022] GET /identity/v3/groups/2ba4933eecf34efe9993645362010421 => generated 226 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:04.186183 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc3749-9548-48e8-9e3d-3bbbaa7dc9e8 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:04.255190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea87032-62e4-4233-852e-bb7d5fe42d5f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:04.328084 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3dc51e-0a4b-4e0c-9dec-d8edf3ab7595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:04.329218 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dc51e-0a4b-4e0c-9dec-d8edf3ab7595 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:04.329699 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dc51e-0a4b-4e0c-9dec-d8edf3ab7595 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:04.330168 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3dc51e-0a4b-4e0c-9dec-d8edf3ab7595 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:04.331150 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3dc51e-0a4b-4e0c-9dec-d8edf3ab7595 None admin] RBAC: Adding query filter params (name=member) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:04.331649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3dc51e-0a4b-4e0c-9dec-d8edf3ab7595 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:04.333314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3dc51e-0a4b-4e0c-9dec-d8edf3ab7595 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:04.357807 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 215/426] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:14:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:14:04.361662 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be28b0-a5fa-4d92-a92b-e1dbadcb6df3 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:04.362985 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be28b0-a5fa-4d92-a92b-e1dbadcb6df3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:04.363472 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be28b0-a5fa-4d92-a92b-e1dbadcb6df3 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:04.371737 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 216/427] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:04 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:04.373331 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be28b0-a5fa-4d92-a92b-e1dbadcb6df3 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:04.407294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:04.441394 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc3749-9548-48e8-9e3d-3bbbaa7dc9e8 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:04.548428 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e75ed8b-c971-4a78-b4b2-bac6d98f21d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:04.549456 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75ed8b-c971-4a78-b4b2-bac6d98f21d8 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:04.551402 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75ed8b-c971-4a78-b4b2-bac6d98f21d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:04.552264 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e75ed8b-c971-4a78-b4b2-bac6d98f21d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:04.555609 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75ed8b-c971-4a78-b4b2-bac6d98f21d8 None admin] RBAC: Adding query filter params (group.id=2ba4933eecf34efe9993645362010421, role.id=31034f95ef88402296c753c2c259f717, scope.project.id=95abfdd95158427e9f9f0571955df6c4) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:04.556221 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75ed8b-c971-4a78-b4b2-bac6d98f21d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:04.559061 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be28b0-a5fa-4d92-a92b-e1dbadcb6df3 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:04.560192 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e75ed8b-c971-4a78-b4b2-bac6d98f21d8 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:04.575970 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 212/428] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jun 30 10:14:04 2022] GET /identity/v3/role_assignments?group.id=2ba4933eecf34efe9993645362010421&scope.project.id=95abfdd95158427e9f9f0571955df6c4&role.id=31034f95ef88402296c753c2c259f717 => generated 611 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:04.578949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202793f-185b-498b-890d-9f75db805534 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:04.579582 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202793f-185b-498b-890d-9f75db805534 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:04.580058 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9202793f-185b-498b-890d-9f75db805534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:04.598644 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3de259-270a-4932-8876-38f3980fc797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:04.599339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3de259-270a-4932-8876-38f3980fc797 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:04.599631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3de259-270a-4932-8876-38f3980fc797 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:04.599909 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3de259-270a-4932-8876-38f3980fc797 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:04.622844 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-9202793f-185b-498b-890d-9f75db805534 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:04.624936 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-9202793f-185b-498b-890d-9f75db805534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:04.674582 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3de259-270a-4932-8876-38f3980fc797 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:04.676748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3de259-270a-4932-8876-38f3980fc797 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:04.693639 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:04.722578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:04.823693 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 213/429] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:04.825083 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 217/430] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:14:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:14:04.830690 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202793f-185b-498b-890d-9f75db805534 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:04.848820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3de259-270a-4932-8876-38f3980fc797 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:04.968139 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9202793f-185b-498b-890d-9f75db805534 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:04.974598 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3de259-270a-4932-8876-38f3980fc797 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:04.992587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:05.016027 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dca421b-4373-4daa-a9ee-9e55d223c7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:05.016913 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca421b-4373-4daa-a9ee-9e55d223c7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:05.017191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca421b-4373-4daa-a9ee-9e55d223c7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:05.017465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dca421b-4373-4daa-a9ee-9e55d223c7b8 None admin] PATH_INFO: `/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:05.019516 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca421b-4373-4daa-a9ee-9e55d223c7b8 None admin] RBAC: Authorizing `identity:get_project(project_id=47885b7cd44548ce92d8cfb4b55c8a97)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:05.020984 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dca421b-4373-4daa-a9ee-9e55d223c7b8 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:05.023124 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 214/431] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:14:04 2022] GET /identity/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:05.027443 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bcfb3c-f3d0-4132-b1de-a51cabc942da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:05.027582 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e6272-ba83-41b6-a254-972d67d0327e None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:05.027681 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcfb3c-f3d0-4132-b1de-a51cabc942da None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:05.027754 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e6272-ba83-41b6-a254-972d67d0327e None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:05.027821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcfb3c-f3d0-4132-b1de-a51cabc942da None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:05.027887 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58e6272-ba83-41b6-a254-972d67d0327e None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:05.027952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bcfb3c-f3d0-4132-b1de-a51cabc942da None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:05.028111 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 215/432] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:05.029527 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e6272-ba83-41b6-a254-972d67d0327e None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:05.041400 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bcfb3c-f3d0-4132-b1de-a51cabc942da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:05.050594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-51bcfb3c-f3d0-4132-b1de-a51cabc942da None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 10:14:05.050594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:05.050594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:05.050594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:05.050594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:05.050594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:05.050594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:05.050594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:05.050594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:05.050594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:05.050594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:05.050594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:05.050594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:05.050594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:05.050594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:05.050594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:05.051515 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:05.051515 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:05.051515 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:05.051515 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:05.051515 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:05.051515 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:05.051515 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:05.051515 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:05.051515 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:05.051515 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:05.051515 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:05.051515 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:05.051515 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:05.051515 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:05.051515 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:05.051515 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:05.052316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:05.052316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:05.052316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:05.052316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:05.052316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:05.052316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:05.052316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:05.052316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:05.052316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:05.052316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:05.052316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 10:14:05.052316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:05.057135 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 218/433] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jun 30 10:14:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:05.058161 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25193c-e806-448b-844a-7d2ed05a6c78 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:05.058673 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25193c-e806-448b-844a-7d2ed05a6c78 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:05.058774 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25193c-e806-448b-844a-7d2ed05a6c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:05.076927 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0f25193c-e806-448b-844a-7d2ed05a6c78 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:05.077879 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:05.078183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0f25193c-e806-448b-844a-7d2ed05a6c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:05.145313 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58e6272-ba83-41b6-a254-972d67d0327e None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:05.150942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 219/434] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:05.156129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25193c-e806-448b-844a-7d2ed05a6c78 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:05.184793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27582d77-51aa-467b-8264-3f2b078ebee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:05.185739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27582d77-51aa-467b-8264-3f2b078ebee9 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:05.186029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27582d77-51aa-467b-8264-3f2b078ebee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:05.186377 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27582d77-51aa-467b-8264-3f2b078ebee9 None admin] PATH_INFO: `/v3/groups/2ba4933eecf34efe9993645362010421` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:05.187691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27582d77-51aa-467b-8264-3f2b078ebee9 None admin] RBAC: Authorizing `identity:get_group(group_id=2ba4933eecf34efe9993645362010421)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:05.189131 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27582d77-51aa-467b-8264-3f2b078ebee9 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:05.190891 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 216/435] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jun 30 10:14:05 2022] GET /identity/v3/groups/2ba4933eecf34efe9993645362010421 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:14:05.191680 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27582d77-51aa-467b-8264-3f2b078ebee9 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:05.275035 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25193c-e806-448b-844a-7d2ed05a6c78 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:05.277533 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:05.297275 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27582d77-51aa-467b-8264-3f2b078ebee9 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:05.305071 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:05.309607 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ecda3b-f598-4e3f-8510-29ec81fcddb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:05.310273 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecda3b-f598-4e3f-8510-29ec81fcddb9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:05.310555 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecda3b-f598-4e3f-8510-29ec81fcddb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:05.310828 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ecda3b-f598-4e3f-8510-29ec81fcddb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:05.311607 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ecda3b-f598-4e3f-8510-29ec81fcddb9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:05.311913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ecda3b-f598-4e3f-8510-29ec81fcddb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:05.313398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ecda3b-f598-4e3f-8510-29ec81fcddb9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:05.323304 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 220/436] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:05.324104 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ecda3b-f598-4e3f-8510-29ec81fcddb9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:05.339342 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a8e7b-88b3-449e-8bc5-29dfccab89c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:05.340165 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a8e7b-88b3-449e-8bc5-29dfccab89c5 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:05.340459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a8e7b-88b3-449e-8bc5-29dfccab89c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:05.340721 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78a8e7b-88b3-449e-8bc5-29dfccab89c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:05.351984 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78a8e7b-88b3-449e-8bc5-29dfccab89c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:05.360824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-b78a8e7b-88b3-449e-8bc5-29dfccab89c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 10:14:05.360824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:05.360824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:05.360824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:05.360824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:05.360824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:05.360824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:05.360824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:05.360824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:05.360824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:05.360824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:05.360824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:05.360824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:05.360824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:05.360824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:05.360824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:05.361783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:05.361783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:05.361783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:05.361783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:05.361783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:05.361783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:05.361783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:05.361783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:05.361783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:05.361783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:05.361783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:05.361783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:05.361783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:05.361783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:05.361783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:05.361783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:05.363444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:05.363444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:05.363444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:05.363444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:05.363444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:05.363444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:05.363444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:05.363444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:05.363444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:05.363444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:05.363444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 10:14:05.363444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:05.363444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 217/437] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jun 30 10:14:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:05.363444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a8e7b-88b3-449e-8bc5-29dfccab89c5 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:05.429777 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ecda3b-f598-4e3f-8510-29ec81fcddb9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:05.462876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78a8e7b-88b3-449e-8bc5-29dfccab89c5 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:05.463916 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad7691-8b88-4600-944b-c6bea4690f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:05.464579 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad7691-8b88-4600-944b-c6bea4690f9b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:05.464864 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad7691-8b88-4600-944b-c6bea4690f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:05.465153 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad7691-8b88-4600-944b-c6bea4690f9b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:05.465985 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad7691-8b88-4600-944b-c6bea4690f9b None admin] RBAC: Adding query filter params (group.id=2ba4933eecf34efe9993645362010421, role.id=31034f95ef88402296c753c2c259f717, scope.project.id=47885b7cd44548ce92d8cfb4b55c8a97) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:05.466301 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad7691-8b88-4600-944b-c6bea4690f9b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:05.467793 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdad7691-8b88-4600-944b-c6bea4690f9b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:05.473544 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 221/438] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jun 30 10:14:05 2022] GET /identity/v3/role_assignments?group.id=2ba4933eecf34efe9993645362010421&scope.project.id=47885b7cd44548ce92d8cfb4b55c8a97&role.id=31034f95ef88402296c753c2c259f717 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:05.474744 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a282a-2f3a-4e8e-a0b6-fccadce86f27 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:05.475038 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a282a-2f3a-4e8e-a0b6-fccadce86f27 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:05.475343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a282a-2f3a-4e8e-a0b6-fccadce86f27 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:05.476598 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 222/439] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:05.477322 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a282a-2f3a-4e8e-a0b6-fccadce86f27 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:05.509549 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d62e2f-f4c9-4a75-9c83-50f9c6105e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:05.510232 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d62e2f-f4c9-4a75-9c83-50f9c6105e82 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:05.510515 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d62e2f-f4c9-4a75-9c83-50f9c6105e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:05.510796 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d62e2f-f4c9-4a75-9c83-50f9c6105e82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:05.520243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:05.551786 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:05.592475 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d62e2f-f4c9-4a75-9c83-50f9c6105e82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:05.594345 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d62e2f-f4c9-4a75-9c83-50f9c6105e82 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:05.602142 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a282a-2f3a-4e8e-a0b6-fccadce86f27 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:05.640237 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6b763c-4cdb-462d-88a2-6c75f3380c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:05.640917 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b763c-4cdb-462d-88a2-6c75f3380c73 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:05.641206 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b763c-4cdb-462d-88a2-6c75f3380c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:05.641482 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6b763c-4cdb-462d-88a2-6c75f3380c73 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:05.642240 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b763c-4cdb-462d-88a2-6c75f3380c73 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:05.642550 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b763c-4cdb-462d-88a2-6c75f3380c73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:05.644503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6b763c-4cdb-462d-88a2-6c75f3380c73 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:05.653888 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 223/440] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:05.655108 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a60593c-f5ac-4b2e-bc1f-91ad3e2c1830 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:05.656361 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a60593c-f5ac-4b2e-bc1f-91ad3e2c1830 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:05.656644 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a60593c-f5ac-4b2e-bc1f-91ad3e2c1830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:05.678311 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8a60593c-f5ac-4b2e-bc1f-91ad3e2c1830 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:05.679589 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8a60593c-f5ac-4b2e-bc1f-91ad3e2c1830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:05.736460 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 218/441] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:14:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:14:05.737426 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d62e2f-f4c9-4a75-9c83-50f9c6105e82 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:05.769355 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 224/442] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:05.770234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a60593c-f5ac-4b2e-bc1f-91ad3e2c1830 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:05.796922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:05.822874 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:05.863896 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d62e2f-f4c9-4a75-9c83-50f9c6105e82 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:05.883652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a60593c-f5ac-4b2e-bc1f-91ad3e2c1830 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:05.899691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd399b-c4ed-4f90-863d-ecdf51096725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:05.900347 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd399b-c4ed-4f90-863d-ecdf51096725 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:05.900634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd399b-c4ed-4f90-863d-ecdf51096725 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:05.900909 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd399b-c4ed-4f90-863d-ecdf51096725 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:05.919192 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697dc2bb-17f0-4b38-b3e4-17116ebe42a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:05.920088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dc2bb-17f0-4b38-b3e4-17116ebe42a7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:05.920381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dc2bb-17f0-4b38-b3e4-17116ebe42a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:05.920661 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dc2bb-17f0-4b38-b3e4-17116ebe42a7 None admin] PATH_INFO: `/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:05.921888 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697dc2bb-17f0-4b38-b3e4-17116ebe42a7 None admin] RBAC: Authorizing `identity:get_project(project_id=47885b7cd44548ce92d8cfb4b55c8a97)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:05.923381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697dc2bb-17f0-4b38-b3e4-17116ebe42a7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:05.925687 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 225/443] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:14:05 2022] GET /identity/v3/projects/47885b7cd44548ce92d8cfb4b55c8a97 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:05.926889 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697dc2bb-17f0-4b38-b3e4-17116ebe42a7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:05.969863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dd399b-c4ed-4f90-863d-ecdf51096725 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:05.971370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dd399b-c4ed-4f90-863d-ecdf51096725 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:06.014391 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:06.023985 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697dc2bb-17f0-4b38-b3e4-17116ebe42a7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:06.034714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:06.063290 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccab1f9-36c1-44a9-9071-c5232a74a82d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:06.064107 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccab1f9-36c1-44a9-9071-c5232a74a82d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:06.064396 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccab1f9-36c1-44a9-9071-c5232a74a82d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:06.064677 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccab1f9-36c1-44a9-9071-c5232a74a82d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:06.073683 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 219/444] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:14:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:14:06.074585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd399b-c4ed-4f90-863d-ecdf51096725 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:06.075114 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccab1f9-36c1-44a9-9071-c5232a74a82d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:06.083156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-eccab1f9-36c1-44a9-9071-c5232a74a82d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:14:06.083156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:06.083156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:06.083156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:06.083156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:06.083156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:06.083156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:06.083156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:06.083156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:06.083156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:06.083156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:06.083156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:06.083156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:06.083156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:06.083156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:06.083156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:06.084140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:06.084140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:06.084140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:06.084140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:06.084140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:06.084140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:06.084140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:06.084140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:06.084140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:06.084140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:06.084140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:06.084140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:06.084140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:06.084140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:06.084140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:06.084140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:06.085025 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:06.085025 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:06.085025 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:06.085025 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:06.085025 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:06.085025 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:06.085025 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:06.085025 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:06.085025 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:06.085025 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:06.085025 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:14:06.085025 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:06.085025 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 226/445] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:14:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:06.085680 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccab1f9-36c1-44a9-9071-c5232a74a82d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:06.173989 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dd399b-c4ed-4f90-863d-ecdf51096725 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:06.188716 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccab1f9-36c1-44a9-9071-c5232a74a82d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:06.208704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78649a-1cf2-446c-a17a-172bf73d5bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:06.209555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78649a-1cf2-446c-a17a-172bf73d5bcd None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:06.209837 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78649a-1cf2-446c-a17a-172bf73d5bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:06.210111 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c78649a-1cf2-446c-a17a-172bf73d5bcd None admin] PATH_INFO: `/v3/groups/2ba4933eecf34efe9993645362010421` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:06.211426 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c78649a-1cf2-446c-a17a-172bf73d5bcd None admin] RBAC: Authorizing `identity:get_group(group_id=2ba4933eecf34efe9993645362010421)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:06.212858 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c78649a-1cf2-446c-a17a-172bf73d5bcd None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:06.214464 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 220/446] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jun 30 10:14:06 2022] GET /identity/v3/groups/2ba4933eecf34efe9993645362010421 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:06.215220 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78649a-1cf2-446c-a17a-172bf73d5bcd None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:06.222658 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc23468-49e9-4559-979c-c5deb24e2ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:06.223550 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc23468-49e9-4559-979c-c5deb24e2ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:06.223836 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc23468-49e9-4559-979c-c5deb24e2ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:06.224112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc23468-49e9-4559-979c-c5deb24e2ea9 None admin] PATH_INFO: `/v3/projects/95abfdd95158427e9f9f0571955df6c4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:06.225301 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc23468-49e9-4559-979c-c5deb24e2ea9 None admin] RBAC: Authorizing `identity:get_project(project_id=95abfdd95158427e9f9f0571955df6c4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:06.226746 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc23468-49e9-4559-979c-c5deb24e2ea9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:06.228467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 227/447] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:14:06 2022] GET /identity/v3/projects/95abfdd95158427e9f9f0571955df6c4 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:06.229217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc23468-49e9-4559-979c-c5deb24e2ea9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:06.248224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:06.254417 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:06.320273 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c78649a-1cf2-446c-a17a-172bf73d5bcd None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:06.327927 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc23468-49e9-4559-979c-c5deb24e2ea9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:06.353631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0a53e-bd60-4eb9-952b-797a55e4cacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:06.354299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0a53e-bd60-4eb9-952b-797a55e4cacf None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:06.354584 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0a53e-bd60-4eb9-952b-797a55e4cacf None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:06.354859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c0a53e-bd60-4eb9-952b-797a55e4cacf None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:06.355637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0a53e-bd60-4eb9-952b-797a55e4cacf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:06.355943 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0a53e-bd60-4eb9-952b-797a55e4cacf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:06.357311 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c0a53e-bd60-4eb9-952b-797a55e4cacf None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:06.361646 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676c0188-8ade-4607-853e-d9b5bf2e9702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:06.362850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c0188-8ade-4607-853e-d9b5bf2e9702 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:06.363137 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c0188-8ade-4607-853e-d9b5bf2e9702 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:06.363456 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c0188-8ade-4607-853e-d9b5bf2e9702 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:06.364292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676c0188-8ade-4607-853e-d9b5bf2e9702 None admin] RBAC: Adding query filter params (group.id=2ba4933eecf34efe9993645362010421, role.id=284d1573e06d47eba0196afbe92d4d7d, scope.project.id=47885b7cd44548ce92d8cfb4b55c8a97) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:06.364604 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676c0188-8ade-4607-853e-d9b5bf2e9702 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:06.365833 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 221/448] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:14:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:06.366037 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676c0188-8ade-4607-853e-d9b5bf2e9702 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:06.366684 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0a53e-bd60-4eb9-952b-797a55e4cacf None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:06.383293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 228/449] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jun 30 10:14:06 2022] GET /identity/v3/role_assignments?group.id=2ba4933eecf34efe9993645362010421&scope.project.id=47885b7cd44548ce92d8cfb4b55c8a97&role.id=284d1573e06d47eba0196afbe92d4d7d => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:06.383293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676c0188-8ade-4607-853e-d9b5bf2e9702 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:06.473655 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:06.486670 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:06.493492 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c0a53e-bd60-4eb9-952b-797a55e4cacf None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:06.501382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676c0188-8ade-4607-853e-d9b5bf2e9702 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:06.536995 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15bfe33-441a-4d18-8324-74b6cce78722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:06.537862 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bfe33-441a-4d18-8324-74b6cce78722 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:06.538149 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bfe33-441a-4d18-8324-74b6cce78722 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:06.538429 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15bfe33-441a-4d18-8324-74b6cce78722 None admin] PATH_INFO: `/v3/groups/2ba4933eecf34efe9993645362010421` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:06.539640 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15bfe33-441a-4d18-8324-74b6cce78722 None admin] RBAC: Authorizing `identity:get_group(group_id=2ba4933eecf34efe9993645362010421)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:06.541080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15bfe33-441a-4d18-8324-74b6cce78722 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:06.542477 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000de822-e1bf-4366-a117-de4d4353dc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:06.547313 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 222/450] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jun 30 10:14:06 2022] GET /identity/v3/groups/2ba4933eecf34efe9993645362010421 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:06.547446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000de822-e1bf-4366-a117-de4d4353dc20 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:06.547446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000de822-e1bf-4366-a117-de4d4353dc20 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:06.547446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000de822-e1bf-4366-a117-de4d4353dc20 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:06.548737 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15bfe33-441a-4d18-8324-74b6cce78722 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:06.627511 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000de822-e1bf-4366-a117-de4d4353dc20 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:06.629794 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000de822-e1bf-4366-a117-de4d4353dc20 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:06.672979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15bfe33-441a-4d18-8324-74b6cce78722 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:06.711345 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f9928-4f73-42bc-8d56-73233b30882b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:06.712009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f9928-4f73-42bc-8d56-73233b30882b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:06.712289 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f9928-4f73-42bc-8d56-73233b30882b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:06.712561 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f9928-4f73-42bc-8d56-73233b30882b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:06.713451 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f9928-4f73-42bc-8d56-73233b30882b None admin] RBAC: Adding query filter params (group.id=2ba4933eecf34efe9993645362010421, role.id=284d1573e06d47eba0196afbe92d4d7d, scope.project.id=95abfdd95158427e9f9f0571955df6c4) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:06.713768 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f9928-4f73-42bc-8d56-73233b30882b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:06.715189 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f9928-4f73-42bc-8d56-73233b30882b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:06.721996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 223/451] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jun 30 10:14:06 2022] GET /identity/v3/role_assignments?group.id=2ba4933eecf34efe9993645362010421&scope.project.id=95abfdd95158427e9f9f0571955df6c4&role.id=284d1573e06d47eba0196afbe92d4d7d => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:06.739246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:06.758260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 229/452] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:14:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:14:06.764331 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f9928-4f73-42bc-8d56-73233b30882b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:06.805459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:06.895382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f9928-4f73-42bc-8d56-73233b30882b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:06.931418 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6b986-02e1-4496-be97-a9bd7a9be768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:06.932290 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6b986-02e1-4496-be97-a9bd7a9be768 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:06.932567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6b986-02e1-4496-be97-a9bd7a9be768 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:06.932848 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6b986-02e1-4496-be97-a9bd7a9be768 None admin] PATH_INFO: `/v3/projects/429c0e046ffb41cbb44280918656fef0` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:06.934083 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6b986-02e1-4496-be97-a9bd7a9be768 None admin] RBAC: Authorizing `identity:get_project(project_id=429c0e046ffb41cbb44280918656fef0)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:06.935555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6b986-02e1-4496-be97-a9bd7a9be768 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:06.937436 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 224/453] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:14:06 2022] GET /identity/v3/projects/429c0e046ffb41cbb44280918656fef0 => generated 344 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:06.943703 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000de822-e1bf-4366-a117-de4d4353dc20 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:07.048084 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000de822-e1bf-4366-a117-de4d4353dc20 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:07.084114 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a72d3c-3829-4367-8da1-6b6aaf8be5ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:07.084983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a72d3c-3829-4367-8da1-6b6aaf8be5ae None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:07.085269 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a72d3c-3829-4367-8da1-6b6aaf8be5ae None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:07.085543 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a72d3c-3829-4367-8da1-6b6aaf8be5ae None admin] PATH_INFO: `/v3/groups/d238919d52d44211be02a5e1688bc3e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:07.086732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a72d3c-3829-4367-8da1-6b6aaf8be5ae None admin] RBAC: Authorizing `identity:get_group(group_id=d238919d52d44211be02a5e1688bc3e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:07.088230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a72d3c-3829-4367-8da1-6b6aaf8be5ae None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:07.090169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 230/454] 10.222.0.47 () {56 vars in 1180 bytes} [Thu Jun 30 10:14:06 2022] GET /identity/v3/groups/d238919d52d44211be02a5e1688bc3e0 => generated 229 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:07.095890 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6b986-02e1-4496-be97-a9bd7a9be768 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:07.197050 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:07.202552 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6b986-02e1-4496-be97-a9bd7a9be768 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:07.241729 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae51e49-0223-4c5a-aaf0-2536bda8a683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:07.242374 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae51e49-0223-4c5a-aaf0-2536bda8a683 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:07.242657 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae51e49-0223-4c5a-aaf0-2536bda8a683 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:07.242934 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae51e49-0223-4c5a-aaf0-2536bda8a683 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:07.243785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae51e49-0223-4c5a-aaf0-2536bda8a683 None admin] RBAC: Adding query filter params (group.id=d238919d52d44211be02a5e1688bc3e0, role.id=e63bda042e104a4a93fba15ffcfe1183, scope.project.id=429c0e046ffb41cbb44280918656fef0) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:07.244100 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae51e49-0223-4c5a-aaf0-2536bda8a683 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:07.245524 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae51e49-0223-4c5a-aaf0-2536bda8a683 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:07.251161 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 225/455] 10.222.0.47 () {56 vars in 1376 bytes} [Thu Jun 30 10:14:07 2022] GET /identity/v3/role_assignments?group.id=d238919d52d44211be02a5e1688bc3e0&scope.project.id=429c0e046ffb41cbb44280918656fef0&role.id=e63bda042e104a4a93fba15ffcfe1183 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:09.877434 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65c60d-fcf3-433d-9fa5-4d7a1a22c1f9 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:09.879354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65c60d-fcf3-433d-9fa5-4d7a1a22c1f9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:09.879823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65c60d-fcf3-433d-9fa5-4d7a1a22c1f9 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:09.881632 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 231/456] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:09.886922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738fa89-7e63-40fb-8200-30081d08d6dd None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:09.887751 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738fa89-7e63-40fb-8200-30081d08d6dd None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:09.888201 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3738fa89-7e63-40fb-8200-30081d08d6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:09.907341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-3738fa89-7e63-40fb-8200-30081d08d6dd None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:09.908555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-3738fa89-7e63-40fb-8200-30081d08d6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:09.991549 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f5219-835c-4812-baa7-5d0d8a9a8aa8 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:09.992199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f5219-835c-4812-baa7-5d0d8a9a8aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:09.992692 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f5219-835c-4812-baa7-5d0d8a9a8aa8 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:09.994259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 232/457] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:09.998450 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 226/458] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:10.017523 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97681ddf-99ab-450a-9273-a0979768c35f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:10.018117 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97681ddf-99ab-450a-9273-a0979768c35f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:10.019199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97681ddf-99ab-450a-9273-a0979768c35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:10.024794 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738fa89-7e63-40fb-8200-30081d08d6dd None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:10.050038 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-97681ddf-99ab-450a-9273-a0979768c35f None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:10.051530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-97681ddf-99ab-450a-9273-a0979768c35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:10.146274 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:10.188588 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 233/459] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:10.190142 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c74fa-1370-4f15-ba5f-6e3f2e288468 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:10.190944 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c74fa-1370-4f15-ba5f-6e3f2e288468 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:10.191218 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864c74fa-1370-4f15-ba5f-6e3f2e288468 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:10.192641 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 234/460] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:10.210576 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6523e3-bdd8-44a6-b43c-e583a6faf0cd None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:10.210870 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6523e3-bdd8-44a6-b43c-e583a6faf0cd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:10.211205 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6523e3-bdd8-44a6-b43c-e583a6faf0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:10.216464 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:10.223196 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3738fa89-7e63-40fb-8200-30081d08d6dd None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:10.232222 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-bc6523e3-bdd8-44a6-b43c-e583a6faf0cd None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:10.234032 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bc6523e3-bdd8-44a6-b43c-e583a6faf0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:10.263013 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca422ac-ca87-4ce4-943a-38cace80d953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:10.263837 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca422ac-ca87-4ce4-943a-38cace80d953 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:10.264123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca422ac-ca87-4ce4-943a-38cace80d953 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:10.264400 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca422ac-ca87-4ce4-943a-38cace80d953 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:10.275901 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca422ac-ca87-4ce4-943a-38cace80d953 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:10.277454 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca422ac-ca87-4ce4-943a-38cace80d953 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:10.304665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-bca422ac-ca87-4ce4-943a-38cace80d953 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:10.304665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:10.304665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:10.304665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:10.304665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:10.304665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:10.304665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:10.304665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:10.304665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:10.304665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:10.304665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:10.304665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:10.304665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:10.304665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:10.304665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:10.305643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:10.305643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:10.305643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:10.305643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:10.305643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:10.305643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:10.305643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:10.305643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:10.305643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:10.305643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:10.305643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:10.305643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:10.305643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:10.305643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:10.305643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:10.305643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:10.305643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:10.306484 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:10.306484 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:10.306484 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:10.306484 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:10.307096 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 227/461] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:14:10.309417 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca422ac-ca87-4ce4-943a-38cace80d953 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:10.314137 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 235/462] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:10.315291 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6523e3-bdd8-44a6-b43c-e583a6faf0cd None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:10.441001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6523e3-bdd8-44a6-b43c-e583a6faf0cd None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:10.448412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca422ac-ca87-4ce4-943a-38cace80d953 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:10.472145 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:10.486021 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55883450-a855-49eb-896a-30c6ded44b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:10.487022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55883450-a855-49eb-896a-30c6ded44b06 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:10.487607 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55883450-a855-49eb-896a-30c6ded44b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:10.487965 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50775b39-a42b-4cfd-85d9-6d0f43959bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:10.488313 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55883450-a855-49eb-896a-30c6ded44b06 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:10.488771 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50775b39-a42b-4cfd-85d9-6d0f43959bc4 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:10.489056 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50775b39-a42b-4cfd-85d9-6d0f43959bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:10.489333 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50775b39-a42b-4cfd-85d9-6d0f43959bc4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:10.489930 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55883450-a855-49eb-896a-30c6ded44b06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:10.490430 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55883450-a855-49eb-896a-30c6ded44b06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:10.492689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55883450-a855-49eb-896a-30c6ded44b06 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:10.500544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50775b39-a42b-4cfd-85d9-6d0f43959bc4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:10.505256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50775b39-a42b-4cfd-85d9-6d0f43959bc4 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:10.508365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:10.513039 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 236/463] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:10.513039 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55883450-a855-49eb-896a-30c6ded44b06 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:10.535101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-50775b39-a42b-4cfd-85d9-6d0f43959bc4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:10.535101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:10.535101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:10.535101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:10.535101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:10.535101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:10.535101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:10.535101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:10.535101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:10.535101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:10.535101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:10.535101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:10.535101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:10.535101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:10.535101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:10.536064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:10.536064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:10.536064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:10.536064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:10.536064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:10.536064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:10.536064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:10.536064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:10.536064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:10.536064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:10.536064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:10.536064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:10.536064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:10.536064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:10.536064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:10.536064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:10.536064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:10.536902 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:10.536902 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:10.536902 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:10.536902 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:10.538872 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 228/464] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:10.545430 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50775b39-a42b-4cfd-85d9-6d0f43959bc4 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:10.629782 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55883450-a855-49eb-896a-30c6ded44b06 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:10.652719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50775b39-a42b-4cfd-85d9-6d0f43959bc4 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:10.664509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062a5fd-abce-4105-9897-e74e5fa2bc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:10.665303 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062a5fd-abce-4105-9897-e74e5fa2bc06 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:10.665583 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062a5fd-abce-4105-9897-e74e5fa2bc06 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:10.665855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062a5fd-abce-4105-9897-e74e5fa2bc06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:10.677172 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3062a5fd-abce-4105-9897-e74e5fa2bc06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:10.678788 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3062a5fd-abce-4105-9897-e74e5fa2bc06 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:10.689872 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d753f444-9c56-43a5-8d96-b2858dfc202b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:10.691662 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753f444-9c56-43a5-8d96-b2858dfc202b None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:10.692084 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753f444-9c56-43a5-8d96-b2858dfc202b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:10.692543 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d753f444-9c56-43a5-8d96-b2858dfc202b None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:10.693825 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d753f444-9c56-43a5-8d96-b2858dfc202b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:10.696447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d753f444-9c56-43a5-8d96-b2858dfc202b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:10.697916 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-3062a5fd-abce-4105-9897-e74e5fa2bc06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:10.697916 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:10.697916 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:10.697916 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:10.697916 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:10.697916 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:10.697916 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:10.697916 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:10.697916 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:10.697916 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:10.697916 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:10.697916 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:10.697916 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:10.697916 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:10.697916 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:10.698735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:10.698735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:10.698735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:10.698735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:10.698735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:10.698735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:10.698735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:10.698735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:10.698735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:10.698735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:10.698735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:10.698735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:10.698735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:10.698735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:10.698735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:10.698735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:10.698735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:10.699595 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:10.699595 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:10.699595 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:10.699595 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:10.703603 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 237/465] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:10.704724 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062a5fd-abce-4105-9897-e74e5fa2bc06 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:10.733723 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:10.802329 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 229/466] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jun 30 10:14:10 2022] POST /identity/v3/users => generated 246 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:14:10.803530 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ec621-0d3d-477d-b39b-7d50bc15a2db None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:10.803823 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ec621-0d3d-477d-b39b-7d50bc15a2db None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:10.804099 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ec621-0d3d-477d-b39b-7d50bc15a2db None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:10.805617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 230/467] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:10.811066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ec621-0d3d-477d-b39b-7d50bc15a2db None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:10.824056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062a5fd-abce-4105-9897-e74e5fa2bc06 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:10.869889 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f790007-ae5b-4847-ae29-fbb57bb6216c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:10.870920 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f790007-ae5b-4847-ae29-fbb57bb6216c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:10.871426 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f790007-ae5b-4847-ae29-fbb57bb6216c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:10.872011 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f790007-ae5b-4847-ae29-fbb57bb6216c None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:10.872935 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f790007-ae5b-4847-ae29-fbb57bb6216c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:10.873476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f790007-ae5b-4847-ae29-fbb57bb6216c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:10.875095 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f790007-ae5b-4847-ae29-fbb57bb6216c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:10.886442 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:10.889942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 238/468] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:10.891930 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37beebdf-486e-44be-bbef-5c583e6b8652 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:10.892422 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37beebdf-486e-44be-bbef-5c583e6b8652 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:10.892967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37beebdf-486e-44be-bbef-5c583e6b8652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:10.912769 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-37beebdf-486e-44be-bbef-5c583e6b8652 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:10.914310 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-37beebdf-486e-44be-bbef-5c583e6b8652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:10.957282 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ec621-0d3d-477d-b39b-7d50bc15a2db None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:10.995405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 239/469] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:10.996348 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37beebdf-486e-44be-bbef-5c583e6b8652 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:11.012758 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:11.020078 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778cf5a-c387-40ba-a2be-f28aa4cd34d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:11.020775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778cf5a-c387-40ba-a2be-f28aa4cd34d4 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:11.021071 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778cf5a-c387-40ba-a2be-f28aa4cd34d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:11.022848 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778cf5a-c387-40ba-a2be-f28aa4cd34d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:11.022848 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7778cf5a-c387-40ba-a2be-f28aa4cd34d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:11.022848 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7778cf5a-c387-40ba-a2be-f28aa4cd34d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:11.024180 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7778cf5a-c387-40ba-a2be-f28aa4cd34d4 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:11.044167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 231/470] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:11.045922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df72d1-db73-4dad-943c-1bb504c5f6fa None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:11.046448 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df72d1-db73-4dad-943c-1bb504c5f6fa None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:11.046956 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04df72d1-db73-4dad-943c-1bb504c5f6fa None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:11.049097 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 232/471] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:11.050727 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df72d1-db73-4dad-943c-1bb504c5f6fa None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:11.114787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37beebdf-486e-44be-bbef-5c583e6b8652 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:11.150485 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5427f7e3-33ab-4ef5-817e-d318bdb5d109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:11.151154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427f7e3-33ab-4ef5-817e-d318bdb5d109 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:11.151466 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427f7e3-33ab-4ef5-817e-d318bdb5d109 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:11.151741 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5427f7e3-33ab-4ef5-817e-d318bdb5d109 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:11.152443 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5427f7e3-33ab-4ef5-817e-d318bdb5d109 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:11.153890 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5427f7e3-33ab-4ef5-817e-d318bdb5d109 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:11.168412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04df72d1-db73-4dad-943c-1bb504c5f6fa None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:11.180919 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:11.204024 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1374e-5dd2-4291-85f1-f6392c34cb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:11.204820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1374e-5dd2-4291-85f1-f6392c34cb3e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:11.205106 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1374e-5dd2-4291-85f1-f6392c34cb3e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:11.205422 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1374e-5dd2-4291-85f1-f6392c34cb3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:11.215618 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf1374e-5dd2-4291-85f1-f6392c34cb3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:11.217047 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf1374e-5dd2-4291-85f1-f6392c34cb3e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:11.234988 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-eaf1374e-5dd2-4291-85f1-f6392c34cb3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:11.234988 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:11.234988 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:11.234988 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:11.234988 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:11.234988 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:11.234988 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:11.234988 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:11.234988 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:11.234988 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:11.234988 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:11.234988 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:11.234988 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:11.234988 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:11.234988 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:11.236118 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:11.236118 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:11.236118 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:11.236118 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:11.236118 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:11.236118 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:11.236118 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:11.236118 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:11.236118 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:11.236118 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:11.236118 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:11.236118 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:11.236118 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:11.236118 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:11.236118 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:11.236118 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:11.236118 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:11.238319 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:11.238319 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:11.238319 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:11.238319 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:11.238319 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 233/472] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:11.239516 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1374e-5dd2-4291-85f1-f6392c34cb3e None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:11.308778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 240/473] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jun 30 10:14:10 2022] POST /identity/v3/users => generated 248 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:14:11.310078 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7ffa4-12fc-4e2c-8232-da3acc581232 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:11.310369 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7ffa4-12fc-4e2c-8232-da3acc581232 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:11.310646 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc7ffa4-12fc-4e2c-8232-da3acc581232 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:11.342028 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-dbc7ffa4-12fc-4e2c-8232-da3acc581232 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:11.344714 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-dbc7ffa4-12fc-4e2c-8232-da3acc581232 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:11.354647 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1374e-5dd2-4291-85f1-f6392c34cb3e None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:11.402462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3f251-86be-4536-b162-a4b390aec885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:11.403031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3f251-86be-4536-b162-a4b390aec885 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:11.403357 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3f251-86be-4536-b162-a4b390aec885 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:11.403641 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3f251-86be-4536-b162-a4b390aec885 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:11.404385 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3f251-86be-4536-b162-a4b390aec885 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:11.405864 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3f251-86be-4536-b162-a4b390aec885 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:11.435809 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:11.442424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 241/474] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:11.443606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc7ffa4-12fc-4e2c-8232-da3acc581232 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:11.535146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 234/475] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jun 30 10:14:11 2022] POST /identity/v3/users => generated 248 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:14:11.536504 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3f251-86be-4536-b162-a4b390aec885 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:11.560003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:11.562961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc7ffa4-12fc-4e2c-8232-da3acc581232 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:11.604625 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc98c374-e3da-4bbf-bbf2-bf6f49da9ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:11.605360 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98c374-e3da-4bbf-bbf2-bf6f49da9ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:11.605657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98c374-e3da-4bbf-bbf2-bf6f49da9ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:11.605926 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98c374-e3da-4bbf-bbf2-bf6f49da9ca7 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:11.606735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc98c374-e3da-4bbf-bbf2-bf6f49da9ca7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:11.607038 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc98c374-e3da-4bbf-bbf2-bf6f49da9ca7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:11.608509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc98c374-e3da-4bbf-bbf2-bf6f49da9ca7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:11.623647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 242/476] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:11.630282 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc98c374-e3da-4bbf-bbf2-bf6f49da9ca7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:11.653944 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3f251-86be-4536-b162-a4b390aec885 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:11.691952 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa90ea-8082-4605-b48b-f48a45952e64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:11.692666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa90ea-8082-4605-b48b-f48a45952e64 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:11.692935 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa90ea-8082-4605-b48b-f48a45952e64 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:11.693216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6aa90ea-8082-4605-b48b-f48a45952e64 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:11.693973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa90ea-8082-4605-b48b-f48a45952e64 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:11.695448 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6aa90ea-8082-4605-b48b-f48a45952e64 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:11.730956 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.sql.core [None req-b6aa90ea-8082-4605-b48b-f48a45952e64 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 10:14:11.730956 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 30 10:14:11.730956 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [parameters: {'id': '54e44cb332e94719ad868ae5f1f1fea1', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 30 10:14:11.730956 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80364) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 10:14:11.749011 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:11.751441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-b6aa90ea-8082-4605-b48b-f48a45952e64 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 30 10:14:11.751441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:11.751441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 10:14:11.751441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 10:14:11.751441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 10:14:11.751441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 10:14:11.751441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 10:14:11.751441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 10:14:11.751441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 10:14:11.751441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application conn.query(q) Jun 30 10:14:11.751441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 10:14:11.751441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 10:14:11.751441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 10:14:11.751441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application result.read() Jun 30 10:14:11.751441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 10:14:11.752510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 10:14:11.752510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 10:14:11.752510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 10:14:11.752510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 10:14:11.752510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 10:14:11.752510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 10:14:11.752510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 10:14:11.752510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 10:14:11.752510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:11.752510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 10:14:11.752510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:11.752510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:11.752510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 10:14:11.752510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 10:14:11.752510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 30 10:14:11.752510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return ref.to_dict() Jun 30 10:14:11.752510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 10:14:11.752510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application next(self.gen) Jun 30 10:14:11.753829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 10:14:11.753829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application yield resource Jun 30 10:14:11.753829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 10:14:11.753829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application next(self.gen) Jun 30 10:14:11.753829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 10:14:11.753829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 10:14:11.753829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 10:14:11.753829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 10:14:11.753829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 10:14:11.753829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise self.value Jun 30 10:14:11.753829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 10:14:11.753829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 10:14:11.753829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 10:14:11.753829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application session.commit() Jun 30 10:14:11.753829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 10:14:11.753829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 10:14:11.753829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 10:14:11.754954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 10:14:11.754954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 10:14:11.754954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self.session.flush() Jun 30 10:14:11.754954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 10:14:11.754954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 10:14:11.754954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 10:14:11.754954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 10:14:11.754954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 10:14:11.754954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application compat.raise_( Jun 30 10:14:11.754954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 10:14:11.754954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception Jun 30 10:14:11.754954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 10:14:11.754954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 10:14:11.754954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 10:14:11.754954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 10:14:11.754954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 10:14:11.754954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 10:14:11.756187 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 10:14:11.756187 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 10:14:11.756187 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 30 10:14:11.756187 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 30 10:14:11.756187 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 10:14:11.756187 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 10:14:11.756187 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 10:14:11.756187 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 10:14:11.756187 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 10:14:11.756187 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 10:14:11.756187 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 10:14:11.756187 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 10:14:11.756187 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 10:14:11.756187 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 10:14:11.756187 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 10:14:11.756187 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception Jun 30 10:14:11.757195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 10:14:11.757195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 10:14:11.757195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 10:14:11.757195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 10:14:11.757195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 10:14:11.757195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 10:14:11.757195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 10:14:11.757195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application conn.query(q) Jun 30 10:14:11.757195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 10:14:11.757195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 10:14:11.757195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 10:14:11.757195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application result.read() Jun 30 10:14:11.757195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 10:14:11.757195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 10:14:11.757195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 10:14:11.757195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 10:14:11.758211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 10:14:11.758211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 10:14:11.758211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 10:14:11.758211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 10:14:11.758211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 30 10:14:11.758211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 30 10:14:11.758211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [parameters: {'id': '54e44cb332e94719ad868ae5f1f1fea1', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 30 10:14:11.758211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 10:14:11.758211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:11.758211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:14:11.758211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:11.758211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:11.758211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:11.758211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:11.758211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:11.758211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:11.759341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:11.759341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:11.759341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:11.759341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:11.759341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:11.759341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:11.759341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 30 10:14:11.759341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 30 10:14:11.759341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:11.759341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:11.759341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 30 10:14:11.759341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 30 10:14:11.759341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 10:14:11.759341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 10:14:11.759341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 30 10:14:11.759341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:11.760358 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 235/477] 10.222.0.47 () {60 vars in 1131 bytes} [Thu Jun 30 10:14:11 2022] POST /identity/v3/roles => generated 144 bytes in 217 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:11.761254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa90ea-8082-4605-b48b-f48a45952e64 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:11.809566 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc98c374-e3da-4bbf-bbf2-bf6f49da9ca7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:11.848717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35201b72-ccfe-4220-8d4e-59c1e5b398f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:11.849683 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35201b72-ccfe-4220-8d4e-59c1e5b398f4 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:11.850162 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35201b72-ccfe-4220-8d4e-59c1e5b398f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:11.850689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35201b72-ccfe-4220-8d4e-59c1e5b398f4 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:11.851570 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35201b72-ccfe-4220-8d4e-59c1e5b398f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:11.853352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35201b72-ccfe-4220-8d4e-59c1e5b398f4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:11.857966 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:11.880146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6aa90ea-8082-4605-b48b-f48a45952e64 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:11.922862 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 243/478] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jun 30 10:14:11 2022] POST /identity/v3/users => generated 249 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:14:11.925957 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03c962e-154f-46b6-be85-b25c748ca05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:11.926804 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c962e-154f-46b6-be85-b25c748ca05f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:11.927088 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c962e-154f-46b6-be85-b25c748ca05f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:11.927391 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03c962e-154f-46b6-be85-b25c748ca05f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:11.946328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03c962e-154f-46b6-be85-b25c748ca05f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:11.955461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-e03c962e-154f-46b6-be85-b25c748ca05f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 10:14:11.955461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:11.955461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:11.955461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:11.955461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:11.955461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:11.955461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:11.955461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:11.955461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:11.955461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:11.955461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:11.955461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:11.955461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:11.955461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:11.955461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:11.955461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:11.956909 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:11.956909 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:11.956909 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:11.956909 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:11.956909 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:11.956909 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:11.956909 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:11.956909 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:11.956909 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:11.956909 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:11.956909 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:11.956909 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:11.956909 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:11.956909 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:11.956909 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:11.956909 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:11.958633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:11.958633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:11.958633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:11.958633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:11.958633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:11.958633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:11.958633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:11.958633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:11.958633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:11.958633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:11.958633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 30 10:14:11.958633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:11.962724 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 236/479] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jun 30 10:14:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:11.968585 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35201b72-ccfe-4220-8d4e-59c1e5b398f4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:12.084044 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35201b72-ccfe-4220-8d4e-59c1e5b398f4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:12.099998 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:12.121142 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5673c-776c-4bab-9fcb-b3be60150dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:12.121830 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5673c-776c-4bab-9fcb-b3be60150dd3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:12.122119 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5673c-776c-4bab-9fcb-b3be60150dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:12.122399 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5673c-776c-4bab-9fcb-b3be60150dd3 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:12.123160 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f5673c-776c-4bab-9fcb-b3be60150dd3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:12.123504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f5673c-776c-4bab-9fcb-b3be60150dd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:12.124910 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f5673c-776c-4bab-9fcb-b3be60150dd3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:12.134927 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 244/480] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:13.342830 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c0528-2c6c-4c45-bd03-7cbfc0f995e5 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:13.343370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c0528-2c6c-4c45-bd03-7cbfc0f995e5 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:13.343480 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032c0528-2c6c-4c45-bd03-7cbfc0f995e5 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:13.345124 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 237/481] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:13.350619 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e990ea-12d0-438f-b105-ddbca8d3025f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:13.351019 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e990ea-12d0-438f-b105-ddbca8d3025f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:13.351183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e990ea-12d0-438f-b105-ddbca8d3025f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:13.379426 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-36e990ea-12d0-438f-b105-ddbca8d3025f None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:13.380699 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-36e990ea-12d0-438f-b105-ddbca8d3025f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:13.472999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 245/482] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 10:14:13.516746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c0528-2c6c-4c45-bd03-7cbfc0f995e5 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:13.641050 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032c0528-2c6c-4c45-bd03-7cbfc0f995e5 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:13.682773 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f7e31-ef9c-4a7a-b4c5-2c6fefd81fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:13.683613 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f7e31-ef9c-4a7a-b4c5-2c6fefd81fdd None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:13.683903 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f7e31-ef9c-4a7a-b4c5-2c6fefd81fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:13.684185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467f7e31-ef9c-4a7a-b4c5-2c6fefd81fdd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:13.695786 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467f7e31-ef9c-4a7a-b4c5-2c6fefd81fdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:13.703586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-467f7e31-ef9c-4a7a-b4c5-2c6fefd81fdd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:13.703586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:13.703586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:13.703586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:13.703586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:13.703586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:13.703586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:13.703586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:13.703586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:13.703586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:13.703586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:13.703586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:13.703586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:13.703586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:13.703586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:13.703586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:13.704510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:13.704510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:13.704510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:13.704510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:13.704510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:13.704510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:13.704510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:13.704510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:13.704510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:13.704510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:13.704510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:13.704510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:13.704510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:13.704510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:13.704510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:13.704510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:13.705310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:13.705310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:13.705310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:13.705310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:13.705310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:13.705310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:13.705310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:13.705310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:13.705310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:13.705310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:13.705310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:13.705310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:13.705310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 238/483] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jun 30 10:14:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:13.710753 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e990ea-12d0-438f-b105-ddbca8d3025f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:13.838240 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e990ea-12d0-438f-b105-ddbca8d3025f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:13.850029 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:13.888045 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a7238-e7b3-40ee-8437-a82dcfeda37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:13.888726 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a7238-e7b3-40ee-8437-a82dcfeda37a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:13.889016 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a7238-e7b3-40ee-8437-a82dcfeda37a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:13.889299 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a7238-e7b3-40ee-8437-a82dcfeda37a None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:13.890061 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a7238-e7b3-40ee-8437-a82dcfeda37a None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:13.890374 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a7238-e7b3-40ee-8437-a82dcfeda37a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:13.891854 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a7238-e7b3-40ee-8437-a82dcfeda37a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:13.905496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 246/484] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jun 30 10:14:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:13.911404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f7e31-ef9c-4a7a-b4c5-2c6fefd81fdd None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:13.914283 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:13.991150 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828865d-24ae-4c53-bfe9-b0e3669f121e None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:13.991519 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828865d-24ae-4c53-bfe9-b0e3669f121e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:13.991755 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4828865d-24ae-4c53-bfe9-b0e3669f121e None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:13.993134 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 247/485] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:14:14.004540 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac86ed-509a-4b76-84ba-a93dc6d9e55e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:14.004832 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac86ed-509a-4b76-84ba-a93dc6d9e55e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:14.005117 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac86ed-509a-4b76-84ba-a93dc6d9e55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:14.029690 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-26ac86ed-509a-4b76-84ba-a93dc6d9e55e None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:14.030895 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-26ac86ed-509a-4b76-84ba-a93dc6d9e55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:14.064645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467f7e31-ef9c-4a7a-b4c5-2c6fefd81fdd None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:14.134197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1b26e-d2a6-41a7-a747-d5c3a4f7fe1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:14.134999 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1b26e-d2a6-41a7-a747-d5c3a4f7fe1d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:14.135299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1b26e-d2a6-41a7-a747-d5c3a4f7fe1d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:14.135581 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1b26e-d2a6-41a7-a747-d5c3a4f7fe1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:14.137896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 248/486] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:14.150028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1b26e-d2a6-41a7-a747-d5c3a4f7fe1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:14.151591 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1b26e-d2a6-41a7-a747-d5c3a4f7fe1d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:14.169743 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-02d1b26e-d2a6-41a7-a747-d5c3a4f7fe1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:14.169743 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:14.169743 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:14.169743 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:14.169743 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:14.169743 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:14.169743 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:14.169743 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:14.169743 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:14.169743 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:14.169743 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:14.169743 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:14.169743 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:14.169743 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:14.169743 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:14.170646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:14.170646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:14.170646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:14.170646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:14.170646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:14.170646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:14.170646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:14.170646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:14.170646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:14.170646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:14.170646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:14.170646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:14.170646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:14.170646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:14.170646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:14.170646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:14.170646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:14.171497 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:14.171497 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:14.171497 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:14.171497 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:14.171497 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 239/487] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:14.178284 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac86ed-509a-4b76-84ba-a93dc6d9e55e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:14.181150 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1b26e-d2a6-41a7-a747-d5c3a4f7fe1d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:14.248352 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:14.297538 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1b26e-d2a6-41a7-a747-d5c3a4f7fe1d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:14.354313 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac86ed-509a-4b76-84ba-a93dc6d9e55e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:14.363209 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b661d2fb-35d2-4d21-ae56-51b2515df8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:14.364049 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661d2fb-35d2-4d21-ae56-51b2515df8b6 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:14.364332 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661d2fb-35d2-4d21-ae56-51b2515df8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:14.364608 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661d2fb-35d2-4d21-ae56-51b2515df8b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:14.370452 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:14.377000 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b661d2fb-35d2-4d21-ae56-51b2515df8b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:14.384971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-b661d2fb-35d2-4d21-ae56-51b2515df8b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:14.384971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:14.384971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:14.384971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:14.384971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:14.384971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:14.384971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:14.384971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:14.384971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:14.384971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:14.384971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:14.384971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:14.384971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:14.384971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:14.384971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:14.384971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:14.385996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:14.385996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:14.385996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:14.385996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:14.385996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:14.385996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:14.385996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:14.385996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:14.385996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:14.385996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:14.385996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:14.385996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:14.385996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:14.385996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:14.385996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:14.385996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:14.386787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:14.386787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:14.386787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:14.386787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:14.386787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:14.386787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:14.386787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:14.386787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:14.386787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:14.386787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:14.386787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:14.386787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:14.386787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 240/488] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jun 30 10:14:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:14.388225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e3529-561f-444f-9a33-9f0156f48928 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:14.388225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e3529-561f-444f-9a33-9f0156f48928 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:14.388225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e3529-561f-444f-9a33-9f0156f48928 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:14.393550 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 241/489] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:14.394942 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078094a-7797-40fa-adc0-d8c1b6abeeff None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:14.395232 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078094a-7797-40fa-adc0-d8c1b6abeeff None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:14.395542 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078094a-7797-40fa-adc0-d8c1b6abeeff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:14.413630 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ffb86-3ac5-45d2-8428-3a23e815164d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:14.414625 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ffb86-3ac5-45d2-8428-3a23e815164d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:14.415005 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ffb86-3ac5-45d2-8428-3a23e815164d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:14.415508 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255ffb86-3ac5-45d2-8428-3a23e815164d None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:14.416071 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-c078094a-7797-40fa-adc0-d8c1b6abeeff None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:14.416435 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ffb86-3ac5-45d2-8428-3a23e815164d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:14.416838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ffb86-3ac5-45d2-8428-3a23e815164d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:14.417745 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-c078094a-7797-40fa-adc0-d8c1b6abeeff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:14.418451 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255ffb86-3ac5-45d2-8428-3a23e815164d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:14.433524 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 249/490] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:14.434494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ffb86-3ac5-45d2-8428-3a23e815164d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:14.492840 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 242/491] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:14.494027 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078094a-7797-40fa-adc0-d8c1b6abeeff None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:14.544280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:14.563682 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255ffb86-3ac5-45d2-8428-3a23e815164d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:14.597568 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd78d126-63ba-4b89-be06-38f35cbc15fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:14.598235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78d126-63ba-4b89-be06-38f35cbc15fd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:14.598516 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78d126-63ba-4b89-be06-38f35cbc15fd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:14.598793 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78d126-63ba-4b89-be06-38f35cbc15fd None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:14.599565 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd78d126-63ba-4b89-be06-38f35cbc15fd None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:14.599873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd78d126-63ba-4b89-be06-38f35cbc15fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:14.601260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd78d126-63ba-4b89-be06-38f35cbc15fd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:14.605723 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078094a-7797-40fa-adc0-d8c1b6abeeff None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:14.609095 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 250/492] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jun 30 10:14:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:14.609876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd78d126-63ba-4b89-be06-38f35cbc15fd None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:14.635633 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:14.643711 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371fcc70-c352-4395-8867-3c177d43aec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:14.644719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fcc70-c352-4395-8867-3c177d43aec5 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:14.645144 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fcc70-c352-4395-8867-3c177d43aec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:14.645595 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371fcc70-c352-4395-8867-3c177d43aec5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:14.653329 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371fcc70-c352-4395-8867-3c177d43aec5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:14.654791 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371fcc70-c352-4395-8867-3c177d43aec5 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:14.670801 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-371fcc70-c352-4395-8867-3c177d43aec5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 10:14:14.670801 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:14.670801 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:14.670801 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:14.670801 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:14.670801 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:14.670801 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:14.670801 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:14.670801 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:14.670801 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:14.670801 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:14.670801 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:14.670801 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:14:14.670801 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:14:14.670801 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:14:14.670801 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:14:14.671835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:14.671835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:14.671835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:14:14.671835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:14.671835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:14:14.671835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:14.671835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:14.671835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:14.671835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:14.671835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:14.671835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:14.671835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:14.671835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:14.671835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:14.671835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:14.671835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:14.671835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:14.671835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:14.672731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:14.672731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:14.672731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:14:14.672731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:14:14.672731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:14:14.672731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:14:14.672731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:14:14.672731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:14:14.672731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 10:14:14.672731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:14.672731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 243/493] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jun 30 10:14:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:14.673787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb126ff5-22ab-4f10-b74e-8eff44dd57e5 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:14.673787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb126ff5-22ab-4f10-b74e-8eff44dd57e5 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:14.673944 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb126ff5-22ab-4f10-b74e-8eff44dd57e5 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:14.679944 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 244/494] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:14.681120 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb126ff5-22ab-4f10-b74e-8eff44dd57e5 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:14.724557 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd78d126-63ba-4b89-be06-38f35cbc15fd None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:14.758434 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362cd6d-fd2c-4548-82ae-0ac662c1c006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:14.759217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362cd6d-fd2c-4548-82ae-0ac662c1c006 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:14.759528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362cd6d-fd2c-4548-82ae-0ac662c1c006 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:14.759801 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b362cd6d-fd2c-4548-82ae-0ac662c1c006 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:14.770416 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b362cd6d-fd2c-4548-82ae-0ac662c1c006 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:14.778377 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-b362cd6d-fd2c-4548-82ae-0ac662c1c006 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:14.778377 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:14.778377 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:14.778377 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:14.778377 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:14.778377 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:14.778377 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:14.778377 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:14.778377 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:14.778377 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:14.778377 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:14.778377 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:14.778377 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:14.778377 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:14.778377 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:14.778377 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:14.779494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:14.779494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:14.779494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:14.779494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:14.779494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:14.779494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:14.779494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:14.779494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:14.779494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:14.779494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:14.779494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:14.779494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:14.779494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:14.779494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:14.779494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:14.779494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:14.780401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:14.780401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:14.780401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:14.780401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:14.780401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:14.780401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:14.780401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:14.780401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:14.780401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:14.780401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:14.780401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:14.780401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:14.780401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 251/495] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jun 30 10:14:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:14.780401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362cd6d-fd2c-4548-82ae-0ac662c1c006 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:14.790476 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb126ff5-22ab-4f10-b74e-8eff44dd57e5 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:14.792797 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:14.823811 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfb6bee-c8ba-4890-93ed-71772ad24c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:14.824599 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb6bee-c8ba-4890-93ed-71772ad24c67 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:14.824880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb6bee-c8ba-4890-93ed-71772ad24c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:14.825152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfb6bee-c8ba-4890-93ed-71772ad24c67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:14.836410 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfb6bee-c8ba-4890-93ed-71772ad24c67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:14.837872 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bfb6bee-c8ba-4890-93ed-71772ad24c67 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:14.856195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-1bfb6bee-c8ba-4890-93ed-71772ad24c67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:14.856195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:14.856195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:14.856195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:14.856195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:14.856195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:14.856195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:14.856195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:14.856195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:14.856195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:14.856195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:14.856195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:14.856195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:14.856195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:14.856195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:14.857037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:14.857037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:14.857037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:14.857037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:14.857037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:14.857037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:14.857037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:14.857037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:14.857037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:14.857037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:14.857037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:14.857037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:14.857037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:14.857037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:14.857037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:14.857037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:14.857037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:14.857869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:14.857869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:14.857869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:14.857869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:14.857869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 245/496] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:14.858987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde1a95-c3d5-4f47-82a9-b98061dd079d None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:14.859294 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde1a95-c3d5-4f47-82a9-b98061dd079d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:14.859570 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde1a95-c3d5-4f47-82a9-b98061dd079d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:14.866411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:14.876606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-dfde1a95-c3d5-4f47-82a9-b98061dd079d None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:14.882344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-dfde1a95-c3d5-4f47-82a9-b98061dd079d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:14.885727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b362cd6d-fd2c-4548-82ae-0ac662c1c006 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:14.919594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e5095-46c8-48d5-be4a-0bf0a68714ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:14.920262 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e5095-46c8-48d5-be4a-0bf0a68714ad None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:14.920558 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e5095-46c8-48d5-be4a-0bf0a68714ad None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:14.920892 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654e5095-46c8-48d5-be4a-0bf0a68714ad None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:14.921682 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e5095-46c8-48d5-be4a-0bf0a68714ad None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:14.921997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e5095-46c8-48d5-be4a-0bf0a68714ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:14.923460 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654e5095-46c8-48d5-be4a-0bf0a68714ad None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:14.945322 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 246/497] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:14.946418 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6cc38-a702-4e2c-af5c-226338f75b7f None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:14.946707 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6cc38-a702-4e2c-af5c-226338f75b7f None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:14.947012 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6cc38-a702-4e2c-af5c-226338f75b7f None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:14.948314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 247/498] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:14.949005 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6cc38-a702-4e2c-af5c-226338f75b7f None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:14.968880 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 252/499] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jun 30 10:14:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:14.969751 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e5095-46c8-48d5-be4a-0bf0a68714ad None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:15.041745 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:15.056908 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6cc38-a702-4e2c-af5c-226338f75b7f None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:15.068600 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654e5095-46c8-48d5-be4a-0bf0a68714ad None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:15.091468 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8d7ef-c7af-43dc-a519-c39d6b55a711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:15.092395 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8d7ef-c7af-43dc-a519-c39d6b55a711 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:15.092880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8d7ef-c7af-43dc-a519-c39d6b55a711 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:15.093580 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8d7ef-c7af-43dc-a519-c39d6b55a711 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:15.094327 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb8d7ef-c7af-43dc-a519-c39d6b55a711 None admin] RBAC: Adding query filter params (name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:15.094641 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb8d7ef-c7af-43dc-a519-c39d6b55a711 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:15.096085 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb8d7ef-c7af-43dc-a519-c39d6b55a711 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:15.102178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f6e83-ee54-4cd9-856c-583408008979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:15.102832 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f6e83-ee54-4cd9-856c-583408008979 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:15.103118 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f6e83-ee54-4cd9-856c-583408008979 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:15.103455 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777f6e83-ee54-4cd9-856c-583408008979 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:15.103875 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 248/500] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jun 30 10:14:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:15.104167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f6e83-ee54-4cd9-856c-583408008979 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:15.104475 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f6e83-ee54-4cd9-856c-583408008979 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:15.105123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216d4da-642b-4268-9702-8dcf1dffc8d5 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:15.105411 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216d4da-642b-4268-9702-8dcf1dffc8d5 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:15.105687 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d216d4da-642b-4268-9702-8dcf1dffc8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:15.105870 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777f6e83-ee54-4cd9-856c-583408008979 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:15.114706 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:15.119937 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 253/501] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:15.120794 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f6e83-ee54-4cd9-856c-583408008979 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:15.121679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-d216d4da-642b-4268-9702-8dcf1dffc8d5 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:15.122878 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d216d4da-642b-4268-9702-8dcf1dffc8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:15.193215 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 249/502] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:15.194051 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216d4da-642b-4268-9702-8dcf1dffc8d5 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:15.225199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777f6e83-ee54-4cd9-856c-583408008979 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:15.259605 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee280a95-0b33-410c-b116-5d2b77fb528f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:15.260396 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee280a95-0b33-410c-b116-5d2b77fb528f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:15.260682 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee280a95-0b33-410c-b116-5d2b77fb528f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:15.260959 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee280a95-0b33-410c-b116-5d2b77fb528f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:15.271384 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee280a95-0b33-410c-b116-5d2b77fb528f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:15.279233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-ee280a95-0b33-410c-b116-5d2b77fb528f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:15.279233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:15.279233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:15.279233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:15.279233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:15.279233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:15.279233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:15.279233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:15.279233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:15.279233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:15.279233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:15.279233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:15.279233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:15.279233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:15.279233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:15.279233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:15.280309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:15.280309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:15.280309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:15.280309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:15.280309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:15.280309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:15.280309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:15.280309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:15.280309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:15.280309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:15.280309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:15.280309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:15.280309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:15.280309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:15.280309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:15.280309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:15.281170 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:15.281170 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:15.281170 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:15.281170 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:15.281170 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:15.281170 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:15.281170 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:15.281170 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:15.281170 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:15.281170 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:15.281170 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:15.281170 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:15.281170 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 254/503] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jun 30 10:14:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:15.281822 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee280a95-0b33-410c-b116-5d2b77fb528f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:15.291112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:15.312850 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d216d4da-642b-4268-9702-8dcf1dffc8d5 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:15.338533 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:15.347301 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0863d-8220-468a-bf34-7eb2fda3e61a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:15.347951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0863d-8220-468a-bf34-7eb2fda3e61a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:15.348239 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0863d-8220-468a-bf34-7eb2fda3e61a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:15.348516 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0863d-8220-468a-bf34-7eb2fda3e61a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:15.381952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee280a95-0b33-410c-b116-5d2b77fb528f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:15.414242 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ac454-e0fe-42e8-94c1-126bf2e64b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:15.415029 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ac454-e0fe-42e8-94c1-126bf2e64b1e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:15.415350 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ac454-e0fe-42e8-94c1-126bf2e64b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:15.415634 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4ac454-e0fe-42e8-94c1-126bf2e64b1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:15.416713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0863d-8220-468a-bf34-7eb2fda3e61a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:15.418192 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0863d-8220-468a-bf34-7eb2fda3e61a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:15.426577 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ac454-e0fe-42e8-94c1-126bf2e64b1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:15.428077 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4ac454-e0fe-42e8-94c1-126bf2e64b1e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:15.445640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-ab4ac454-e0fe-42e8-94c1-126bf2e64b1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:15.445640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:15.445640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:15.445640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:15.445640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:15.445640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:15.445640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:15.445640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:15.445640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:15.445640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:15.445640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:15.445640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:15.445640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:15.445640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:15.445640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:15.446553 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:15.446553 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:15.446553 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:15.446553 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:15.446553 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:15.446553 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:15.446553 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:15.446553 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:15.446553 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:15.446553 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:15.446553 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:15.446553 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:15.446553 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:15.446553 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:15.446553 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:15.446553 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:15.446553 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:15.447436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:15.447436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:15.447436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:15.447436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:15.447436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 255/504] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:15.447693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ac454-e0fe-42e8-94c1-126bf2e64b1e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:15.500971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:15.524108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 250/505] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:14:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:14:15.524857 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0863d-8220-468a-bf34-7eb2fda3e61a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:15.547998 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4ac454-e0fe-42e8-94c1-126bf2e64b1e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:15.558610 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:15.581730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1767b-ed91-4a20-9fe2-15d5f9b7a1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:15.582580 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1767b-ed91-4a20-9fe2-15d5f9b7a1b0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:15.582864 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1767b-ed91-4a20-9fe2-15d5f9b7a1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:15.583143 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1767b-ed91-4a20-9fe2-15d5f9b7a1b0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:15.590671 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1767b-ed91-4a20-9fe2-15d5f9b7a1b0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:15.592495 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1767b-ed91-4a20-9fe2-15d5f9b7a1b0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:15.608345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-9cd1767b-ed91-4a20-9fe2-15d5f9b7a1b0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 10:14:15.608345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:15.608345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:15.608345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:15.608345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:15.608345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:15.608345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:15.608345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:15.608345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:15.608345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:15.608345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:15.608345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:15.608345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:14:15.608345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:14:15.608345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:14:15.608345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:14:15.609385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:15.609385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:15.609385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:14:15.609385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:15.609385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:14:15.609385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:15.609385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:15.609385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:15.609385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:15.609385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:15.609385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:15.609385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:15.609385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:15.609385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:15.609385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:15.609385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:15.609385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:15.609385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:15.610275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:15.610275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:15.610275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:14:15.610275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:14:15.610275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:14:15.610275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:14:15.610275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:14:15.610275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:14:15.610275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 10:14:15.610275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:15.610275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 256/506] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:14:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:15.610837 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1767b-ed91-4a20-9fe2-15d5f9b7a1b0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:15.629522 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0863d-8220-468a-bf34-7eb2fda3e61a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:15.664247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb7b46-0d23-4a52-8fee-cbe0a6cfd11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:15.665073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb7b46-0d23-4a52-8fee-cbe0a6cfd11f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:15.665355 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb7b46-0d23-4a52-8fee-cbe0a6cfd11f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:15.665628 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb7b46-0d23-4a52-8fee-cbe0a6cfd11f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:15.676213 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb7b46-0d23-4a52-8fee-cbe0a6cfd11f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:15.677669 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb7b46-0d23-4a52-8fee-cbe0a6cfd11f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:15.694720 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-4fcb7b46-0d23-4a52-8fee-cbe0a6cfd11f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:15.694720 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:15.694720 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:15.694720 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:15.694720 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:15.694720 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:15.694720 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:15.694720 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:15.694720 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:15.694720 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:15.694720 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:15.694720 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:15.694720 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:15.694720 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:15.694720 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:15.695635 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:15.695635 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:15.695635 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:15.695635 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:15.695635 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:15.695635 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:15.695635 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:15.695635 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:15.695635 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:15.695635 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:15.695635 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:15.695635 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:15.695635 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:15.695635 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:15.695635 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:15.695635 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:15.695635 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:15.696518 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:15.696518 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:15.696518 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:15.696518 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:15.696518 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 251/507] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:15.697127 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb7b46-0d23-4a52-8fee-cbe0a6cfd11f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:15.715507 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1767b-ed91-4a20-9fe2-15d5f9b7a1b0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:15.728397 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:15.749913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526e8cd-2e07-4e6c-b965-67ec97c97457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:15.750615 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526e8cd-2e07-4e6c-b965-67ec97c97457 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:15.750904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526e8cd-2e07-4e6c-b965-67ec97c97457 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:15.751187 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b526e8cd-2e07-4e6c-b965-67ec97c97457 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:15.751977 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526e8cd-2e07-4e6c-b965-67ec97c97457 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:15.752329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526e8cd-2e07-4e6c-b965-67ec97c97457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:15.753947 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b526e8cd-2e07-4e6c-b965-67ec97c97457 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:15.761994 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 257/508] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jun 30 10:14:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:15.764107 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526e8cd-2e07-4e6c-b965-67ec97c97457 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:15.796972 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:15.798529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb7b46-0d23-4a52-8fee-cbe0a6cfd11f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:15.833558 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621253e-b013-46a0-943a-7854b23ea77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:15.834221 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621253e-b013-46a0-943a-7854b23ea77b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:15.834503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621253e-b013-46a0-943a-7854b23ea77b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:15.834775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621253e-b013-46a0-943a-7854b23ea77b None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:15.835522 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621253e-b013-46a0-943a-7854b23ea77b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:15.835823 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621253e-b013-46a0-943a-7854b23ea77b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:15.837188 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621253e-b013-46a0-943a-7854b23ea77b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:15.849016 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 252/509] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:15.849804 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621253e-b013-46a0-943a-7854b23ea77b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:15.865828 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b526e8cd-2e07-4e6c-b965-67ec97c97457 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:15.898683 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f525e-c6f5-42a6-b4a2-28019cfcffda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:15.899509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f525e-c6f5-42a6-b4a2-28019cfcffda None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:15.899795 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f525e-c6f5-42a6-b4a2-28019cfcffda None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:15.900074 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2f525e-c6f5-42a6-b4a2-28019cfcffda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:15.910471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2f525e-c6f5-42a6-b4a2-28019cfcffda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:15.911972 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2f525e-c6f5-42a6-b4a2-28019cfcffda None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:15.929814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-bc2f525e-c6f5-42a6-b4a2-28019cfcffda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:15.929814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:15.929814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:15.929814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:15.929814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:15.929814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:15.929814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:15.929814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:15.929814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:15.929814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:15.929814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:15.929814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:15.929814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:15.929814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:15.929814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:15.930843 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:15.930843 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:15.930843 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:15.930843 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:15.930843 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:15.930843 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:15.930843 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:15.930843 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:15.930843 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:15.930843 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:15.930843 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:15.930843 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:15.930843 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:15.930843 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:15.930843 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:15.930843 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:15.930843 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:15.931888 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:15.931888 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:15.931888 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:15.931888 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:15.931888 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 258/510] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:15.932289 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f525e-c6f5-42a6-b4a2-28019cfcffda None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:15.945724 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:15.960492 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621253e-b013-46a0-943a-7854b23ea77b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:15.989762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53468c8-f3c9-48e9-824f-0c798564a806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:15.990434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53468c8-f3c9-48e9-824f-0c798564a806 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:15.990720 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53468c8-f3c9-48e9-824f-0c798564a806 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:15.991000 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53468c8-f3c9-48e9-824f-0c798564a806 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:15.991866 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53468c8-f3c9-48e9-824f-0c798564a806 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:15.992179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53468c8-f3c9-48e9-824f-0c798564a806 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:15.993643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53468c8-f3c9-48e9-824f-0c798564a806 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:16.019914 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:16.031578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 253/511] 10.222.0.47 () {56 vars in 1137 bytes} [Thu Jun 30 10:14:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:16.032428 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53468c8-f3c9-48e9-824f-0c798564a806 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:16.037710 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2f525e-c6f5-42a6-b4a2-28019cfcffda None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:16.071502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e758ce4-4a76-4801-b489-2ef480788501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:16.072159 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e758ce4-4a76-4801-b489-2ef480788501 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:16.072450 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e758ce4-4a76-4801-b489-2ef480788501 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:16.072724 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e758ce4-4a76-4801-b489-2ef480788501 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:16.073439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e758ce4-4a76-4801-b489-2ef480788501 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:16.073746 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e758ce4-4a76-4801-b489-2ef480788501 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:16.075111 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e758ce4-4a76-4801-b489-2ef480788501 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:16.087359 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 259/512] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:16.088146 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e758ce4-4a76-4801-b489-2ef480788501 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:16.131842 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53468c8-f3c9-48e9-824f-0c798564a806 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:16.166607 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d8078-43f1-448c-a021-22fa4fb2a6d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:16.167428 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d8078-43f1-448c-a021-22fa4fb2a6d2 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:16.167710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d8078-43f1-448c-a021-22fa4fb2a6d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:16.167983 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837d8078-43f1-448c-a021-22fa4fb2a6d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:16.178340 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d8078-43f1-448c-a021-22fa4fb2a6d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:16.179834 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837d8078-43f1-448c-a021-22fa4fb2a6d2 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:16.188106 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:16.188952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e758ce4-4a76-4801-b489-2ef480788501 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:16.197624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-837d8078-43f1-448c-a021-22fa4fb2a6d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:16.197624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:16.197624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:16.197624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:16.197624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:16.197624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:16.197624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:16.197624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:16.197624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:16.197624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:16.197624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:16.197624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:16.197624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:16.197624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:16.197624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:16.201529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:16.201529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:16.201529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:16.201529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:16.201529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:16.201529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:16.201529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:16.201529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:16.201529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:16.201529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:16.201529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:16.201529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:16.201529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:16.201529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:16.201529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:16.201529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:16.201529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:16.202585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:16.202585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:16.202585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:16.202585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:16.202585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 254/513] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:16.202585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d8078-43f1-448c-a021-22fa4fb2a6d2 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:16.224742 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a1a75-f976-45e1-b45c-f1cd5e53b585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:16.225465 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a1a75-f976-45e1-b45c-f1cd5e53b585 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:16.225747 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a1a75-f976-45e1-b45c-f1cd5e53b585 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:16.226020 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a1a75-f976-45e1-b45c-f1cd5e53b585 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:16.232966 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a1a75-f976-45e1-b45c-f1cd5e53b585 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:16.234425 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a1a75-f976-45e1-b45c-f1cd5e53b585 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:16.248689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-ae9a1a75-f976-45e1-b45c-f1cd5e53b585 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 10:14:16.248689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:16.248689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:16.248689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:16.248689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:16.248689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:16.248689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:16.248689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:16.248689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:16.248689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:16.248689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:16.248689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:16.248689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:14:16.248689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:14:16.248689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:14:16.248689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:14:16.249652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:16.249652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:16.249652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:14:16.249652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:16.249652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:14:16.249652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:16.249652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:16.249652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:16.249652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:16.249652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:16.249652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:16.249652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:16.249652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:16.249652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:16.249652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:16.249652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:16.249652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:16.249652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:16.250556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:16.250556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:16.250556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:14:16.250556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:14:16.250556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:14:16.250556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:14:16.250556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:14:16.250556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:14:16.250556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 10:14:16.250556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:16.250556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 260/514] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:14:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:16.251133 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a1a75-f976-45e1-b45c-f1cd5e53b585 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:16.255363 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:16.307828 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837d8078-43f1-448c-a021-22fa4fb2a6d2 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:16.342053 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9075e4b-998f-48a8-81e5-52b2f9c3036d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:16.342707 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9075e4b-998f-48a8-81e5-52b2f9c3036d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:16.342986 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9075e4b-998f-48a8-81e5-52b2f9c3036d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:16.343255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9075e4b-998f-48a8-81e5-52b2f9c3036d None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:16.343988 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9075e4b-998f-48a8-81e5-52b2f9c3036d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:16.344286 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9075e4b-998f-48a8-81e5-52b2f9c3036d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:16.345643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9075e4b-998f-48a8-81e5-52b2f9c3036d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:16.351323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a1a75-f976-45e1-b45c-f1cd5e53b585 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:16.357292 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 255/515] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:16.358070 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9075e4b-998f-48a8-81e5-52b2f9c3036d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:16.384995 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0b0a2-c6af-4d53-becf-84e8e2ff7328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:16.385630 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0b0a2-c6af-4d53-becf-84e8e2ff7328 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:16.385932 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0b0a2-c6af-4d53-becf-84e8e2ff7328 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:16.386191 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d0b0a2-c6af-4d53-becf-84e8e2ff7328 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:16.416588 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:16.453667 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0b0a2-c6af-4d53-becf-84e8e2ff7328 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:16.455281 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d0b0a2-c6af-4d53-becf-84e8e2ff7328 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:16.457395 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9075e4b-998f-48a8-81e5-52b2f9c3036d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:16.459834 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:16.503140 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089366ef-eac0-465c-8c4e-5c1fcf0c7d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:16.503655 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089366ef-eac0-465c-8c4e-5c1fcf0c7d5b None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:16.503940 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089366ef-eac0-465c-8c4e-5c1fcf0c7d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:16.504223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089366ef-eac0-465c-8c4e-5c1fcf0c7d5b None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:16.504935 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089366ef-eac0-465c-8c4e-5c1fcf0c7d5b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:16.506370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089366ef-eac0-465c-8c4e-5c1fcf0c7d5b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:16.557987 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 261/516] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:14:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:14:16.558835 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0b0a2-c6af-4d53-becf-84e8e2ff7328 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:16.595055 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 256/517] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jun 30 10:14:16 2022] POST /identity/v3/users => generated 247 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:14:16.595921 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089366ef-eac0-465c-8c4e-5c1fcf0c7d5b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:16.672411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d0b0a2-c6af-4d53-becf-84e8e2ff7328 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:16.675520 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:16.710950 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089366ef-eac0-465c-8c4e-5c1fcf0c7d5b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:16.712092 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab8281-1b8d-4553-9511-3f1c5cc3ef29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:16.712982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab8281-1b8d-4553-9511-3f1c5cc3ef29 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:16.713275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab8281-1b8d-4553-9511-3f1c5cc3ef29 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:16.714456 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab8281-1b8d-4553-9511-3f1c5cc3ef29 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:16.715236 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab8281-1b8d-4553-9511-3f1c5cc3ef29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:16.715595 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab8281-1b8d-4553-9511-3f1c5cc3ef29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:16.717033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab8281-1b8d-4553-9511-3f1c5cc3ef29 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:16.729610 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 262/518] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:16.730485 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab8281-1b8d-4553-9511-3f1c5cc3ef29 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:16.737520 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:16.759502 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda68bb2-f91c-4d44-abb3-3e3d0636eb9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:16.760106 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda68bb2-f91c-4d44-abb3-3e3d0636eb9b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:16.760394 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda68bb2-f91c-4d44-abb3-3e3d0636eb9b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:16.760675 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda68bb2-f91c-4d44-abb3-3e3d0636eb9b None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:16.761518 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda68bb2-f91c-4d44-abb3-3e3d0636eb9b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:16.761828 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda68bb2-f91c-4d44-abb3-3e3d0636eb9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:16.763330 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda68bb2-f91c-4d44-abb3-3e3d0636eb9b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:16.783888 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 257/519] 10.222.0.47 () {56 vars in 1137 bytes} [Thu Jun 30 10:14:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:16.784819 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda68bb2-f91c-4d44-abb3-3e3d0636eb9b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:16.851386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab8281-1b8d-4553-9511-3f1c5cc3ef29 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:16.887057 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26c953-161c-4977-a13f-72117137d247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:16.888237 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26c953-161c-4977-a13f-72117137d247 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:16.888615 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26c953-161c-4977-a13f-72117137d247 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:16.888893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a26c953-161c-4977-a13f-72117137d247 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:16.897232 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26c953-161c-4977-a13f-72117137d247 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:16.898685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a26c953-161c-4977-a13f-72117137d247 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:16.899549 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda68bb2-f91c-4d44-abb3-3e3d0636eb9b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:16.914923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-5a26c953-161c-4977-a13f-72117137d247 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:16.914923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:16.914923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:16.914923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:16.914923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:16.914923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:16.914923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:16.914923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:16.914923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:16.914923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:16.914923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:16.914923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:16.914923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:14:16.914923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:14:16.914923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:14:16.915850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:14:16.915850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:16.915850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:16.915850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:16.915850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:16.915850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:16.915850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:16.915850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:16.915850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:16.915850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:16.915850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:16.915850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:16.915850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:16.915850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:16.915850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:16.915850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:16.915850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:16.916744 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:14:16.916744 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:14:16.916744 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:14:16.916744 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:14:16.916744 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:14:16.916744 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:14:16.916744 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:16.916744 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:16.916744 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 263/520] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:16.917239 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26c953-161c-4977-a13f-72117137d247 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:16.941344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a812a-9b10-4d95-9b6b-0c0a923f7d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:16.942240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a812a-9b10-4d95-9b6b-0c0a923f7d5c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:16.942597 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a812a-9b10-4d95-9b6b-0c0a923f7d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:16.942883 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a812a-9b10-4d95-9b6b-0c0a923f7d5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:16.944095 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:16.953474 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a812a-9b10-4d95-9b6b-0c0a923f7d5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:16.954996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a812a-9b10-4d95-9b6b-0c0a923f7d5c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:16.974254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-d36a812a-9b10-4d95-9b6b-0c0a923f7d5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:16.974254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:16.974254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:16.974254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:16.974254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:16.974254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:16.974254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:16.974254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:16.974254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:16.974254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:16.974254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:16.974254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:16.974254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:16.974254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:16.974254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:16.975185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:16.975185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:16.975185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:16.975185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:16.975185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:16.975185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:16.975185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:16.975185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:16.975185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:16.975185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:16.975185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:16.975185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:16.975185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:16.975185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:16.975185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:16.975185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:16.975185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:16.976242 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:16.976242 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:16.976242 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:16.976242 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:16.976242 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 258/521] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:16.977140 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a812a-9b10-4d95-9b6b-0c0a923f7d5c None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:17.004277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:17.030721 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a26c953-161c-4977-a13f-72117137d247 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:17.068146 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8baff-9a20-4ab3-822f-de5b7b5bb585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:17.068957 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8baff-9a20-4ab3-822f-de5b7b5bb585 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:17.069245 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8baff-9a20-4ab3-822f-de5b7b5bb585 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:17.069526 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8baff-9a20-4ab3-822f-de5b7b5bb585 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:17.076787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8baff-9a20-4ab3-822f-de5b7b5bb585 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:17.078253 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8baff-9a20-4ab3-822f-de5b7b5bb585 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:17.081732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a812a-9b10-4d95-9b6b-0c0a923f7d5c None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:17.093967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-8cb8baff-9a20-4ab3-822f-de5b7b5bb585 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 10:14:17.093967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:17.093967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:17.093967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:17.093967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:17.093967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:17.093967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:17.093967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:17.093967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:17.093967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:17.093967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:17.093967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:17.093967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:14:17.093967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:14:17.093967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:14:17.093967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:14:17.094873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:17.094873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:17.094873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:14:17.094873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:17.094873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:14:17.094873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:17.094873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:17.094873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:17.094873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:17.094873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:17.094873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:17.094873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:17.094873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:17.094873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:17.094873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:17.094873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:17.094873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:17.094873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:17.097129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:17.097129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:17.097129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:14:17.097129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:14:17.097129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:14:17.097129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:14:17.097129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:14:17.097129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:14:17.097129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 10:14:17.097129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:17.097129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 264/522] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jun 30 10:14:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:17.097129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8baff-9a20-4ab3-822f-de5b7b5bb585 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:17.126750 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705a53f-90a3-4a22-8320-b422f2b2e43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:17.127573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705a53f-90a3-4a22-8320-b422f2b2e43c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:17.127856 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705a53f-90a3-4a22-8320-b422f2b2e43c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:17.128145 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705a53f-90a3-4a22-8320-b422f2b2e43c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:17.189388 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:17.201124 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705a53f-90a3-4a22-8320-b422f2b2e43c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:17.202592 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705a53f-90a3-4a22-8320-b422f2b2e43c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:17.208659 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8baff-9a20-4ab3-822f-de5b7b5bb585 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:17.232632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:17.245728 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d6ef2-ab1c-4af1-84c1-c36ac0d4ee94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:17.246397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d6ef2-ab1c-4af1-84c1-c36ac0d4ee94 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:17.246680 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d6ef2-ab1c-4af1-84c1-c36ac0d4ee94 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:17.246959 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d6ef2-ab1c-4af1-84c1-c36ac0d4ee94 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:17.247983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d6ef2-ab1c-4af1-84c1-c36ac0d4ee94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:17.248089 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d6ef2-ab1c-4af1-84c1-c36ac0d4ee94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:17.249529 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d6ef2-ab1c-4af1-84c1-c36ac0d4ee94 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:17.269423 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 265/523] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:14:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:17.270266 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d6ef2-ab1c-4af1-84c1-c36ac0d4ee94 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:17.319236 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 259/524] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:14:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:14:17.320484 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705a53f-90a3-4a22-8320-b422f2b2e43c None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:17.378589 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d6ef2-ab1c-4af1-84c1-c36ac0d4ee94 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:17.419136 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b403caa-408c-4845-b2e5-49548ad43ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:17.420098 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b403caa-408c-4845-b2e5-49548ad43ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:17.420387 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b403caa-408c-4845-b2e5-49548ad43ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:17.420662 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b403caa-408c-4845-b2e5-49548ad43ec5 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:17.421826 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b403caa-408c-4845-b2e5-49548ad43ec5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:17.422158 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b403caa-408c-4845-b2e5-49548ad43ec5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:17.423866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b403caa-408c-4845-b2e5-49548ad43ec5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:17.424436 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705a53f-90a3-4a22-8320-b422f2b2e43c None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:17.435674 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 266/525] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:17.436472 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b403caa-408c-4845-b2e5-49548ad43ec5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:17.442433 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:17.457245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:17.466432 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5b4df-01dd-4bf7-b691-958187c22fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:17.467106 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5b4df-01dd-4bf7-b691-958187c22fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:17.467417 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5b4df-01dd-4bf7-b691-958187c22fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:17.467745 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5b4df-01dd-4bf7-b691-958187c22fe2 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:17.468580 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb5b4df-01dd-4bf7-b691-958187c22fe2 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:17.468890 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb5b4df-01dd-4bf7-b691-958187c22fe2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:17.470324 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb5b4df-01dd-4bf7-b691-958187c22fe2 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:17.492726 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 260/526] 10.222.0.47 () {56 vars in 1139 bytes} [Thu Jun 30 10:14:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:17.493676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5b4df-01dd-4bf7-b691-958187c22fe2 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:17.552544 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b403caa-408c-4845-b2e5-49548ad43ec5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:17.588688 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96445c51-05e7-41b7-b449-f5b0e06903a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:17.589271 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96445c51-05e7-41b7-b449-f5b0e06903a0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:17.589557 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96445c51-05e7-41b7-b449-f5b0e06903a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:17.589840 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96445c51-05e7-41b7-b449-f5b0e06903a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:17.590679 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96445c51-05e7-41b7-b449-f5b0e06903a0 None admin] RBAC: Adding query filter params (role.id=99d8cd9cd87e472290bf0de2710b5857, scope.project.id=50902e0539434c82b505dc5009aeaf85, user.id=71f6d7715e6f49ffb9fdf6e1307bbd51) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:17.591000 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96445c51-05e7-41b7-b449-f5b0e06903a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:17.592524 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96445c51-05e7-41b7-b449-f5b0e06903a0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:17.598567 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 267/527] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:14:17 2022] GET /identity/v3/role_assignments?user.id=71f6d7715e6f49ffb9fdf6e1307bbd51&scope.project.id=50902e0539434c82b505dc5009aeaf85&role.id=99d8cd9cd87e472290bf0de2710b5857 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:17.599359 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5b4df-01dd-4bf7-b691-958187c22fe2 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:17.604316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96445c51-05e7-41b7-b449-f5b0e06903a0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:17.649115 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581cbfae-af11-4c34-98cb-a5d129cffdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:17.650317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cbfae-af11-4c34-98cb-a5d129cffdb1 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:17.650803 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cbfae-af11-4c34-98cb-a5d129cffdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:17.651367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cbfae-af11-4c34-98cb-a5d129cffdb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:17.667928 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581cbfae-af11-4c34-98cb-a5d129cffdb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:17.670763 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581cbfae-af11-4c34-98cb-a5d129cffdb1 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:17.689774 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:17.698970 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-581cbfae-af11-4c34-98cb-a5d129cffdb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:17.698970 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:17.698970 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:17.698970 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:17.698970 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:17.698970 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:17.698970 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:17.698970 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:17.698970 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:17.698970 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:17.698970 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:17.698970 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:17.698970 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:17.698970 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:17.698970 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:17.707216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:17.707216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:17.707216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:17.707216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:17.707216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:17.707216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:17.707216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:17.707216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:17.707216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:17.707216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:17.707216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:17.707216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:17.707216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:17.707216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:17.707216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:17.707216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:17.707216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:17.708109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:17.708109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:17.708109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:17.708109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:17.708109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 261/528] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:17.708109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581cbfae-af11-4c34-98cb-a5d129cffdb1 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:17.729040 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96445c51-05e7-41b7-b449-f5b0e06903a0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:17.748845 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:17.775623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6517b-5bc0-476f-861f-09fbac82734a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:17.776938 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6517b-5bc0-476f-861f-09fbac82734a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:17.777360 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6517b-5bc0-476f-861f-09fbac82734a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:17.777736 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce6517b-5bc0-476f-861f-09fbac82734a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:17.786282 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6517b-5bc0-476f-861f-09fbac82734a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:17.787808 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce6517b-5bc0-476f-861f-09fbac82734a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:17.802424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-2ce6517b-5bc0-476f-861f-09fbac82734a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:17.802424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:17.802424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:17.802424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:17.802424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:17.802424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:17.802424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:17.802424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:17.802424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:17.802424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:17.802424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:17.802424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:17.802424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:14:17.802424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:14:17.802424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:14:17.803375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:14:17.803375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:17.803375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:17.803375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:17.803375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:17.803375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:17.803375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:17.803375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:17.803375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:17.803375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:17.803375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:17.803375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:17.803375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:17.803375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:17.803375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:17.803375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:17.803375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:17.804270 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:14:17.804270 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:14:17.804270 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:14:17.804270 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:14:17.804270 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:14:17.804270 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:14:17.804270 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:17.804270 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:17.804270 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 268/529] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:17.804733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6517b-5bc0-476f-861f-09fbac82734a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:17.848645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581cbfae-af11-4c34-98cb-a5d129cffdb1 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:17.886954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650e620-b2ac-4f10-a0ed-5b6a47040a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:17.888039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650e620-b2ac-4f10-a0ed-5b6a47040a5b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:17.888624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650e620-b2ac-4f10-a0ed-5b6a47040a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:17.889205 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650e620-b2ac-4f10-a0ed-5b6a47040a5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:17.912033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce6517b-5bc0-476f-861f-09fbac82734a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:17.940743 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:17.949332 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85314ac8-d83c-452f-b097-c43814c1e64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:17.950179 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85314ac8-d83c-452f-b097-c43814c1e64c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:17.950361 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85314ac8-d83c-452f-b097-c43814c1e64c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:17.950636 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85314ac8-d83c-452f-b097-c43814c1e64c None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:17.951403 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85314ac8-d83c-452f-b097-c43814c1e64c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:17.951708 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85314ac8-d83c-452f-b097-c43814c1e64c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:17.953191 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85314ac8-d83c-452f-b097-c43814c1e64c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:17.970780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 269/530] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:17.970780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85314ac8-d83c-452f-b097-c43814c1e64c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:17.973507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3650e620-b2ac-4f10-a0ed-5b6a47040a5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:17.975157 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3650e620-b2ac-4f10-a0ed-5b6a47040a5b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:18.013296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:18.084062 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85314ac8-d83c-452f-b097-c43814c1e64c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:18.098589 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 262/531] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:14:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:14:18.098589 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650e620-b2ac-4f10-a0ed-5b6a47040a5b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:18.139505 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33374b49-4ee1-481e-94b5-58031ef431e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:18.140550 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33374b49-4ee1-481e-94b5-58031ef431e1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:18.141035 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33374b49-4ee1-481e-94b5-58031ef431e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:18.141525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33374b49-4ee1-481e-94b5-58031ef431e1 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:18.142537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33374b49-4ee1-481e-94b5-58031ef431e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:18.143249 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33374b49-4ee1-481e-94b5-58031ef431e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:18.145001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33374b49-4ee1-481e-94b5-58031ef431e1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:18.159396 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 270/532] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:14:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:18.167844 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33374b49-4ee1-481e-94b5-58031ef431e1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:18.209960 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650e620-b2ac-4f10-a0ed-5b6a47040a5b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:18.212577 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:18.247172 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51af457-f428-4de4-80fb-be8853550118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:18.248136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51af457-f428-4de4-80fb-be8853550118 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:18.248337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51af457-f428-4de4-80fb-be8853550118 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:18.248591 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51af457-f428-4de4-80fb-be8853550118 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:18.251418 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:18.256599 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51af457-f428-4de4-80fb-be8853550118 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:18.258224 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51af457-f428-4de4-80fb-be8853550118 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:18.268668 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33374b49-4ee1-481e-94b5-58031ef431e1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:18.273744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-d51af457-f428-4de4-80fb-be8853550118 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:18.273744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:18.273744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:18.273744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:18.273744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:18.273744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:18.273744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:18.273744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:18.273744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:18.273744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:18.273744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:18.273744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:18.273744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:14:18.273744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:14:18.273744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:14:18.274762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:14:18.274762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:18.274762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:18.274762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:18.274762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:18.274762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:18.274762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:18.274762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:18.274762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:18.274762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:18.274762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:18.274762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:18.274762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:18.274762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:18.274762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:18.274762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:18.274762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:18.275860 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:14:18.275860 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:14:18.275860 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:14:18.275860 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:14:18.275860 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:14:18.275860 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:14:18.275860 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:18.275860 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:18.275860 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 263/533] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:18.276416 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51af457-f428-4de4-80fb-be8853550118 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:18.309311 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda9da3-648f-4fb7-9ef1-dd423d494796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:18.310553 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda9da3-648f-4fb7-9ef1-dd423d494796 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:18.311180 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda9da3-648f-4fb7-9ef1-dd423d494796 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:18.311583 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda9da3-648f-4fb7-9ef1-dd423d494796 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:18.324607 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda9da3-648f-4fb7-9ef1-dd423d494796 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:18.326888 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda9da3-648f-4fb7-9ef1-dd423d494796 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:18.351178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-1eda9da3-648f-4fb7-9ef1-dd423d494796 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:18.351178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:18.351178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:18.351178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:18.351178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:18.351178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:18.351178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:18.351178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:18.351178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:18.351178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:18.351178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:18.351178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:18.351178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:18.351178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:18.351178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:18.352224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:18.352224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:18.352224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:18.352224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:18.352224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:18.352224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:18.352224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:18.352224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:18.352224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:18.352224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:18.352224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:18.352224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:18.352224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:18.352224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:18.352224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:18.352224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:18.352224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:18.353176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:18.353176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:18.353176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:18.353176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:18.359520 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 271/534] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:18.360506 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda9da3-648f-4fb7-9ef1-dd423d494796 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:18.392004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51af457-f428-4de4-80fb-be8853550118 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:18.441457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe17c98-dab1-4bf8-a2c2-a1c8b2d24609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:18.442486 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe17c98-dab1-4bf8-a2c2-a1c8b2d24609 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:18.443507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe17c98-dab1-4bf8-a2c2-a1c8b2d24609 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:18.443889 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe17c98-dab1-4bf8-a2c2-a1c8b2d24609 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:18.445126 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe17c98-dab1-4bf8-a2c2-a1c8b2d24609 None admin] RBAC: Adding query filter params (role.id=99d8cd9cd87e472290bf0de2710b5857, scope.project.id=50902e0539434c82b505dc5009aeaf85, user.id=c7a61d85908d42698b27b9fc00daa832) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:18.445802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe17c98-dab1-4bf8-a2c2-a1c8b2d24609 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:18.448245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe17c98-dab1-4bf8-a2c2-a1c8b2d24609 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:18.455557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 264/535] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:14:18 2022] GET /identity/v3/role_assignments?user.id=c7a61d85908d42698b27b9fc00daa832&scope.project.id=50902e0539434c82b505dc5009aeaf85&role.id=99d8cd9cd87e472290bf0de2710b5857 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:18.457310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe17c98-dab1-4bf8-a2c2-a1c8b2d24609 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:18.479603 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:18.490012 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda9da3-648f-4fb7-9ef1-dd423d494796 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:18.571665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec5235-7f32-4225-a7ea-952a2262adf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:18.572826 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec5235-7f32-4225-a7ea-952a2262adf5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:18.573485 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec5235-7f32-4225-a7ea-952a2262adf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:18.574689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eec5235-7f32-4225-a7ea-952a2262adf5 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:18.575923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec5235-7f32-4225-a7ea-952a2262adf5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:18.578846 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec5235-7f32-4225-a7ea-952a2262adf5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:18.582117 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eec5235-7f32-4225-a7ea-952a2262adf5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:18.582739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:18.604512 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 272/536] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:14:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:18.614317 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec5235-7f32-4225-a7ea-952a2262adf5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:18.660349 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe17c98-dab1-4bf8-a2c2-a1c8b2d24609 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:18.701712 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca88f6-5e52-4789-a59c-65e288a975d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:18.702741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca88f6-5e52-4789-a59c-65e288a975d2 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:18.703240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca88f6-5e52-4789-a59c-65e288a975d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:18.703753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca88f6-5e52-4789-a59c-65e288a975d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:18.704693 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca88f6-5e52-4789-a59c-65e288a975d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:18.706625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca88f6-5e52-4789-a59c-65e288a975d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:18.708387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bca88f6-5e52-4789-a59c-65e288a975d2 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:18.725767 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 265/537] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:18.731737 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca88f6-5e52-4789-a59c-65e288a975d2 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:18.757233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eec5235-7f32-4225-a7ea-952a2262adf5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:18.794797 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758a86b9-2f63-4105-8412-5af6a15aa612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:18.796059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a86b9-2f63-4105-8412-5af6a15aa612 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:18.796557 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a86b9-2f63-4105-8412-5af6a15aa612 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:18.797080 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a86b9-2f63-4105-8412-5af6a15aa612 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:18.798190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758a86b9-2f63-4105-8412-5af6a15aa612 None admin] RBAC: Adding query filter params (role.id=99d8cd9cd87e472290bf0de2710b5857, scope.project.id=50902e0539434c82b505dc5009aeaf85, user.id=07ab12e5d1d94a009c4d414e5c43ac92) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:18.798795 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758a86b9-2f63-4105-8412-5af6a15aa612 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:18.801169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758a86b9-2f63-4105-8412-5af6a15aa612 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:18.808358 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 273/538] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:14:18 2022] GET /identity/v3/role_assignments?user.id=07ab12e5d1d94a009c4d414e5c43ac92&scope.project.id=50902e0539434c82b505dc5009aeaf85&role.id=99d8cd9cd87e472290bf0de2710b5857 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:18.810297 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8968ba-70e6-4215-9b4e-2db6322fa11c None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:18.810844 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8968ba-70e6-4215-9b4e-2db6322fa11c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:18.811368 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8968ba-70e6-4215-9b4e-2db6322fa11c None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:18.813585 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 274/539] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:18.819619 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa8fd7-2f67-4e62-9cf5-190f18091b4e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:18.820245 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa8fd7-2f67-4e62-9cf5-190f18091b4e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:18.820761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fa8fd7-2f67-4e62-9cf5-190f18091b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:18.856770 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-59fa8fd7-2f67-4e62-9cf5-190f18091b4e None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:18.860342 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-59fa8fd7-2f67-4e62-9cf5-190f18091b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:18.888821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:18.928101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bca88f6-5e52-4789-a59c-65e288a975d2 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:18.966216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:18.980865 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 275/540] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:18.989526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d076906-70e7-4b59-9a30-892d3dd8bc76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:18.990749 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d076906-70e7-4b59-9a30-892d3dd8bc76 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:18.991243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d076906-70e7-4b59-9a30-892d3dd8bc76 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:18.991741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d076906-70e7-4b59-9a30-892d3dd8bc76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:18.999892 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d076906-70e7-4b59-9a30-892d3dd8bc76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:19.002066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d076906-70e7-4b59-9a30-892d3dd8bc76 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:19.012601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa8fd7-2f67-4e62-9cf5-190f18091b4e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:19.033802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-7d076906-70e7-4b59-9a30-892d3dd8bc76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:19.033802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:19.033802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:19.033802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:19.033802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:19.033802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:19.033802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:19.033802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:19.033802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:19.033802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:19.033802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:19.033802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:19.033802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:14:19.033802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:14:19.033802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:14:19.034713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:14:19.034713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:19.034713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:19.034713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:19.034713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:19.034713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:19.034713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:19.034713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:19.034713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:19.034713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:19.034713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:19.034713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:19.034713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:19.034713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:19.034713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:19.034713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:19.034713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:19.035774 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:14:19.035774 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:14:19.035774 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:14:19.035774 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:14:19.035774 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:14:19.035774 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:14:19.035774 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:19.035774 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:19.035774 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 266/541] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:19.044829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d076906-70e7-4b59-9a30-892d3dd8bc76 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:19.148205 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fa8fd7-2f67-4e62-9cf5-190f18091b4e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:19.191128 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77502a-8f37-4e93-982c-3b64533a9ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:19.191785 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77502a-8f37-4e93-982c-3b64533a9ade None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:19.192068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77502a-8f37-4e93-982c-3b64533a9ade None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:19.192342 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77502a-8f37-4e93-982c-3b64533a9ade None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:19.207149 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d076906-70e7-4b59-9a30-892d3dd8bc76 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:19.213622 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db77502a-8f37-4e93-982c-3b64533a9ade None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:19.221436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-db77502a-8f37-4e93-982c-3b64533a9ade None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:19.221436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:19.221436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:19.221436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:19.221436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:19.221436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:19.221436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:19.221436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:19.221436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:19.221436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:19.221436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:19.221436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:19.221436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:19.221436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:19.221436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:19.221436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:19.222353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:19.222353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:19.222353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:19.222353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:19.222353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:19.222353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:19.222353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:19.222353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:19.222353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:19.222353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:19.222353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:19.222353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:19.222353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:19.222353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:19.222353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:19.222353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:19.223164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:19.223164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:19.223164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:19.223164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:19.223164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:19.223164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:19.223164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:19.223164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:19.223164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:19.223164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:19.223164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:19.223164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:19.227725 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 276/542] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jun 30 10:14:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:19.233926 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77502a-8f37-4e93-982c-3b64533a9ade None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:19.267609 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:19.288530 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f5d08-5df7-44b9-800f-bded122cdced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:19.289217 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f5d08-5df7-44b9-800f-bded122cdced None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:19.289531 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f5d08-5df7-44b9-800f-bded122cdced None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:19.294939 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5f5d08-5df7-44b9-800f-bded122cdced None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:19.294939 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5f5d08-5df7-44b9-800f-bded122cdced None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:19.294939 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5f5d08-5df7-44b9-800f-bded122cdced None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:19.294939 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5f5d08-5df7-44b9-800f-bded122cdced None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:19.326929 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 267/543] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:14:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:19.343038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f5d08-5df7-44b9-800f-bded122cdced None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:19.361211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77502a-8f37-4e93-982c-3b64533a9ade None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:19.398545 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450e40e-3c27-4bae-808a-c2cd520b3ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:19.399227 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450e40e-3c27-4bae-808a-c2cd520b3ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:19.399545 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450e40e-3c27-4bae-808a-c2cd520b3ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:19.399822 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450e40e-3c27-4bae-808a-c2cd520b3ff5 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:19.400576 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4450e40e-3c27-4bae-808a-c2cd520b3ff5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:19.400893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4450e40e-3c27-4bae-808a-c2cd520b3ff5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:19.402297 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4450e40e-3c27-4bae-808a-c2cd520b3ff5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:19.411898 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 277/544] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jun 30 10:14:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:19.417745 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450e40e-3c27-4bae-808a-c2cd520b3ff5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:19.456166 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:19.526529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5f5d08-5df7-44b9-800f-bded122cdced None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:19.564797 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2f119-7098-4ef0-9998-ea452988df48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:19.565882 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2f119-7098-4ef0-9998-ea452988df48 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:19.566325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2f119-7098-4ef0-9998-ea452988df48 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:19.566745 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2f119-7098-4ef0-9998-ea452988df48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:19.567748 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2f119-7098-4ef0-9998-ea452988df48 None admin] RBAC: Adding query filter params (role.id=99d8cd9cd87e472290bf0de2710b5857, scope.project.id=50902e0539434c82b505dc5009aeaf85, user.id=059b05007c0144f7a50dbd9f0ef41359) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:19.568221 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2f119-7098-4ef0-9998-ea452988df48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:19.569815 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2f119-7098-4ef0-9998-ea452988df48 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:19.580871 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 268/545] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:14:19 2022] GET /identity/v3/role_assignments?user.id=059b05007c0144f7a50dbd9f0ef41359&scope.project.id=50902e0539434c82b505dc5009aeaf85&role.id=99d8cd9cd87e472290bf0de2710b5857 => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:19.623641 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:19.667823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450e40e-3c27-4bae-808a-c2cd520b3ff5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:19.712549 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffb04d2-e61a-4681-8a0a-d6d1458a4289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:19.713679 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb04d2-e61a-4681-8a0a-d6d1458a4289 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:19.714164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb04d2-e61a-4681-8a0a-d6d1458a4289 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:19.714614 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffb04d2-e61a-4681-8a0a-d6d1458a4289 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:19.726853 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffb04d2-e61a-4681-8a0a-d6d1458a4289 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:19.728670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffb04d2-e61a-4681-8a0a-d6d1458a4289 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:19.748244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-bffb04d2-e61a-4681-8a0a-d6d1458a4289 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:19.748244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:19.748244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:19.748244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:19.748244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:19.748244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:19.748244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:19.748244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:19.748244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:19.748244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:19.748244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:19.748244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:19.748244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:19.748244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:19.748244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:19.749169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:19.749169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:19.749169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:19.749169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:19.749169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:19.749169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:19.749169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:19.749169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:19.749169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:19.749169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:19.749169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:19.749169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:19.749169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:19.749169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:19.749169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:19.749169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:19.749169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:19.750107 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:19.750107 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:19.750107 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:19.750107 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:19.753235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 278/546] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:19.760394 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2f119-7098-4ef0-9998-ea452988df48 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:19.874870 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2f119-7098-4ef0-9998-ea452988df48 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:19.877622 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:19.893443 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99396c-7b06-4593-a3cf-dda96685128b None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:19.893790 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99396c-7b06-4593-a3cf-dda96685128b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:19.896398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d99396c-7b06-4593-a3cf-dda96685128b None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:19.896398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 279/547] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:19.906510 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0d4b4-0935-4d78-ab38-432cb486e08d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:19.906801 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0d4b4-0935-4d78-ab38-432cb486e08d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:19.907059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c0d4b4-0935-4d78-ab38-432cb486e08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:19.929679 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-35c0d4b4-0935-4d78-ab38-432cb486e08d None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:19.931196 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-35c0d4b4-0935-4d78-ab38-432cb486e08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:19.931799 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb6bbe-983f-4a98-8de4-15200cacdbde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:19.932677 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb6bbe-983f-4a98-8de4-15200cacdbde None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:19.933143 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb6bbe-983f-4a98-8de4-15200cacdbde None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:19.933716 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fb6bbe-983f-4a98-8de4-15200cacdbde None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:19.934645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb6bbe-983f-4a98-8de4-15200cacdbde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:19.935128 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb6bbe-983f-4a98-8de4-15200cacdbde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:19.936695 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fb6bbe-983f-4a98-8de4-15200cacdbde None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:19.957644 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 269/548] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:19.964136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb6bbe-983f-4a98-8de4-15200cacdbde None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:20.055846 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 280/549] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:20.075175 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0d4b4-0935-4d78-ab38-432cb486e08d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:20.143991 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:20.191321 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c0d4b4-0935-4d78-ab38-432cb486e08d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:20.224328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fb6bbe-983f-4a98-8de4-15200cacdbde None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:20.230995 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3720442-38e6-449e-8bd4-a45b528ee617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:20.231835 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3720442-38e6-449e-8bd4-a45b528ee617 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:20.232124 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3720442-38e6-449e-8bd4-a45b528ee617 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:20.232403 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3720442-38e6-449e-8bd4-a45b528ee617 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:20.244744 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3720442-38e6-449e-8bd4-a45b528ee617 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:20.257952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-c3720442-38e6-449e-8bd4-a45b528ee617 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:20.257952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:20.257952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:20.257952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:20.257952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:20.257952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:20.257952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:20.257952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:20.257952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:20.257952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:20.257952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:20.257952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:20.257952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:20.257952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:20.257952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:20.257952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:20.258866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:20.258866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:20.258866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:20.258866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:20.258866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:20.258866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:20.258866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:20.258866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:20.258866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:20.258866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:20.258866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:20.258866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:20.258866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:20.258866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:20.258866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:20.258866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:20.259714 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:20.259714 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:20.259714 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:20.259714 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:20.259714 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:20.259714 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:20.259714 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:20.259714 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:20.259714 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:20.259714 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:20.259714 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:20.259714 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:20.259714 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 281/550] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jun 30 10:14:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:14:20.273370 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3720442-38e6-449e-8bd4-a45b528ee617 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:20.302337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aef8c1-ac08-4e82-b748-6843ace59833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:20.303187 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aef8c1-ac08-4e82-b748-6843ace59833 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:20.303675 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aef8c1-ac08-4e82-b748-6843ace59833 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:20.303964 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aef8c1-ac08-4e82-b748-6843ace59833 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:20.311777 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aef8c1-ac08-4e82-b748-6843ace59833 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:20.313667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aef8c1-ac08-4e82-b748-6843ace59833 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:20.329498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-f2aef8c1-ac08-4e82-b748-6843ace59833 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 10:14:20.329498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:20.329498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:20.329498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:20.329498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:20.329498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:20.329498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:20.329498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:20.329498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:20.329498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:20.329498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:20.329498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:20.329498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:14:20.329498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:14:20.329498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:14:20.329498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:14:20.330457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:20.330457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:20.330457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:14:20.330457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:20.330457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:14:20.330457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:20.330457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:20.330457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:20.330457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:20.330457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:20.330457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:20.330457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:20.330457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:20.330457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:20.330457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:20.330457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:20.330457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:20.330457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:20.331434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:20.331434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:20.331434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:14:20.331434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:14:20.331434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:14:20.331434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:14:20.331434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:14:20.331434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:14:20.331434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 10:14:20.331434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:20.331964 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 270/551] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:14:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:20.338259 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aef8c1-ac08-4e82-b748-6843ace59833 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:20.386801 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:20.464621 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3720442-38e6-449e-8bd4-a45b528ee617 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:20.499756 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:20.533521 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a28d22-80d2-4da3-9f73-68ea11fe02ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:20.534710 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a28d22-80d2-4da3-9f73-68ea11fe02ef None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:20.535934 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a28d22-80d2-4da3-9f73-68ea11fe02ef None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:20.536579 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a28d22-80d2-4da3-9f73-68ea11fe02ef None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:20.538595 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aef8c1-ac08-4e82-b748-6843ace59833 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:20.539671 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a28d22-80d2-4da3-9f73-68ea11fe02ef None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:20.541618 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a28d22-80d2-4da3-9f73-68ea11fe02ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:20.543226 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a28d22-80d2-4da3-9f73-68ea11fe02ef None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:20.560641 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 282/552] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jun 30 10:14:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:14:20.566588 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a28d22-80d2-4da3-9f73-68ea11fe02ef None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:20.601449 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126c978-00b8-4c36-82bf-9d8163626344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:20.602459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126c978-00b8-4c36-82bf-9d8163626344 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:20.602959 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126c978-00b8-4c36-82bf-9d8163626344 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:20.603488 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d126c978-00b8-4c36-82bf-9d8163626344 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:20.604534 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126c978-00b8-4c36-82bf-9d8163626344 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:20.605098 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126c978-00b8-4c36-82bf-9d8163626344 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:20.606807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d126c978-00b8-4c36-82bf-9d8163626344 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:20.625778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 271/553] 10.222.0.47 () {56 vars in 1135 bytes} [Thu Jun 30 10:14:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:20.632350 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126c978-00b8-4c36-82bf-9d8163626344 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:20.726108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:20.769592 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d126c978-00b8-4c36-82bf-9d8163626344 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:20.821211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a28d22-80d2-4da3-9f73-68ea11fe02ef None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:20.826457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b901b67-67ad-4107-8a10-2cd075de91db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:20.827413 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b901b67-67ad-4107-8a10-2cd075de91db None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:20.827903 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b901b67-67ad-4107-8a10-2cd075de91db None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:20.828404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b901b67-67ad-4107-8a10-2cd075de91db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:20.905459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b901b67-67ad-4107-8a10-2cd075de91db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:20.907438 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b901b67-67ad-4107-8a10-2cd075de91db None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:20.911879 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17425441-8090-4a6e-8bc9-cbfa0db76d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:20.912963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17425441-8090-4a6e-8bc9-cbfa0db76d8e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:20.913440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17425441-8090-4a6e-8bc9-cbfa0db76d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:20.913904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17425441-8090-4a6e-8bc9-cbfa0db76d8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:20.926736 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17425441-8090-4a6e-8bc9-cbfa0db76d8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:20.929304 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17425441-8090-4a6e-8bc9-cbfa0db76d8e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:20.952486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-17425441-8090-4a6e-8bc9-cbfa0db76d8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:20.952486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:20.952486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:20.952486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:20.952486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:20.952486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:20.952486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:20.952486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:20.952486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:20.952486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:20.952486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:20.952486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:20.952486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:20.952486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:20.952486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:20.953431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:20.953431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:20.953431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:20.953431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:20.953431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:20.953431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:20.953431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:20.953431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:20.953431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:20.953431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:20.953431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:20.953431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:20.953431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:20.953431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:20.953431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:20.953431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:20.953431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:20.954352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:20.954352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:20.954352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:20.954352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:20.954352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 283/554] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 388 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:20.960111 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8548f9-a6a1-4761-91f1-d692adc59bb6 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:20.961281 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8548f9-a6a1-4761-91f1-d692adc59bb6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:20.961557 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8548f9-a6a1-4761-91f1-d692adc59bb6 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:20.962994 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 284/555] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:20.963613 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8548f9-a6a1-4761-91f1-d692adc59bb6 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:21.001668 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:21.027820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:21.047358 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 272/556] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:14:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 10:14:21.048775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f9002-14c3-4e86-b3f7-485b438a57dc None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:21.049117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f9002-14c3-4e86-b3f7-485b438a57dc None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:21.049397 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6f9002-14c3-4e86-b3f7-485b438a57dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:21.067146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-da6f9002-14c3-4e86-b3f7-485b438a57dc None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:21.068482 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-da6f9002-14c3-4e86-b3f7-485b438a57dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:21.100507 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8548f9-a6a1-4761-91f1-d692adc59bb6 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:21.144559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9dccf-39ef-4c0b-ad75-928a96b43b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:21.145363 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9dccf-39ef-4c0b-ad75-928a96b43b74 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:21.145713 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9dccf-39ef-4c0b-ad75-928a96b43b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:21.146094 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9dccf-39ef-4c0b-ad75-928a96b43b74 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:21.147211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe9dccf-39ef-4c0b-ad75-928a96b43b74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:21.147629 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe9dccf-39ef-4c0b-ad75-928a96b43b74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:21.149079 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe9dccf-39ef-4c0b-ad75-928a96b43b74 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:21.158999 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 273/557] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:21.160216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f9002-14c3-4e86-b3f7-485b438a57dc None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:21.169774 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 285/558] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:21.175469 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9dccf-39ef-4c0b-ad75-928a96b43b74 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:21.284735 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6f9002-14c3-4e86-b3f7-485b438a57dc None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:21.288989 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:21.297717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9dccf-39ef-4c0b-ad75-928a96b43b74 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:21.298736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:21.322853 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7259dfb6-0fc8-4bd5-b3e2-a47d58021e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:21.324123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259dfb6-0fc8-4bd5-b3e2-a47d58021e2f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:21.324709 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259dfb6-0fc8-4bd5-b3e2-a47d58021e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:21.325438 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7259dfb6-0fc8-4bd5-b3e2-a47d58021e2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:21.333338 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5303682d-2daa-45f4-976b-81bd83bbd898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:21.334134 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303682d-2daa-45f4-976b-81bd83bbd898 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:21.334421 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303682d-2daa-45f4-976b-81bd83bbd898 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:21.334701 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5303682d-2daa-45f4-976b-81bd83bbd898 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:21.339487 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7259dfb6-0fc8-4bd5-b3e2-a47d58021e2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:21.341037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7259dfb6-0fc8-4bd5-b3e2-a47d58021e2f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:21.342300 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5303682d-2daa-45f4-976b-81bd83bbd898 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:21.343781 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5303682d-2daa-45f4-976b-81bd83bbd898 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:21.358591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-5303682d-2daa-45f4-976b-81bd83bbd898 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 10:14:21.358591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:21.358591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:21.358591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:21.358591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:21.358591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:21.358591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:21.358591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:21.358591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:21.358591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:21.358591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:21.358591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:21.358591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:14:21.358591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:14:21.358591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:14:21.358591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:14:21.359750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:21.359750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:21.359750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:14:21.359750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:21.359750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:14:21.359750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:21.359750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:21.359750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:21.359750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:21.359750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:21.359750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:21.359750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:21.359750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:21.359750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:21.359750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:21.359750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:21.359750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:21.359750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:21.360702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:21.360702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:21.360702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:14:21.360702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:14:21.360702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:14:21.360702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:14:21.360702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:14:21.360702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:14:21.360702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 10:14:21.360702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:21.361233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-7259dfb6-0fc8-4bd5-b3e2-a47d58021e2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:21.361233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:21.361233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:21.361233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:21.361233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:21.361233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:21.361233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:21.361233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:21.361233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:21.361233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:21.361233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:21.361233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:21.361233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:21.361233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:21.361233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:21.362123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:21.362123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:21.362123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:21.362123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:21.362123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:21.362123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:21.362123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:21.362123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:21.362123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:21.362123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:21.362123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:21.362123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:21.362123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:21.362123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:21.362123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:21.362123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:21.362123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:21.364405 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:21.364405 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:21.364405 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:21.364405 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:21.366930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 274/560] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:21.367097 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 286/560] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jun 30 10:14:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:14:21.367555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7259dfb6-0fc8-4bd5-b3e2-a47d58021e2f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:21.368080 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce23332-05e0-40a5-a8c0-a3b03fe820da None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:21.368373 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce23332-05e0-40a5-a8c0-a3b03fe820da None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:21.368652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce23332-05e0-40a5-a8c0-a3b03fe820da None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:21.374318 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 287/561] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:21.374318 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce23332-05e0-40a5-a8c0-a3b03fe820da None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:21.483233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7259dfb6-0fc8-4bd5-b3e2-a47d58021e2f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:21.486470 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce23332-05e0-40a5-a8c0-a3b03fe820da None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:21.521218 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0677c-5b15-4a1d-bb94-a4338863e60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:21.522133 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0677c-5b15-4a1d-bb94-a4338863e60b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:21.522417 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0677c-5b15-4a1d-bb94-a4338863e60b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:21.522691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f0677c-5b15-4a1d-bb94-a4338863e60b None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:21.523567 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0677c-5b15-4a1d-bb94-a4338863e60b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:21.523875 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0677c-5b15-4a1d-bb94-a4338863e60b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:21.525366 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f0677c-5b15-4a1d-bb94-a4338863e60b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:21.527496 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8ac25c-ce8a-49bf-9edf-e1416f08445b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:21.528495 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ac25c-ce8a-49bf-9edf-e1416f08445b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:21.528983 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ac25c-ce8a-49bf-9edf-e1416f08445b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:21.529440 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8ac25c-ce8a-49bf-9edf-e1416f08445b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:21.540717 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8ac25c-ce8a-49bf-9edf-e1416f08445b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:21.544062 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 288/562] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jun 30 10:14:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:21.545129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0677c-5b15-4a1d-bb94-a4338863e60b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:21.548661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-7e8ac25c-ce8a-49bf-9edf-e1416f08445b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:21.548661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:21.548661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:21.548661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:21.548661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:21.548661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:21.548661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:21.548661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:21.548661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:21.548661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:21.548661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:21.548661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:21.548661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:21.548661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:21.548661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:21.548661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:21.550459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:21.550459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:21.550459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:21.550459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:21.550459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:21.550459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:21.550459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:21.550459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:21.550459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:21.550459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:21.550459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:21.550459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:21.550459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:21.550459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:21.550459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:21.550459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:21.551310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:21.551310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:21.551310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:21.551310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:21.551310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:21.551310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:21.551310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:21.551310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:21.551310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:21.551310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:21.551310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:21.551310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:21.555765 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 275/563] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jun 30 10:14:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:21.557087 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954f49f-20a8-4677-9133-92726057acd1 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:21.557375 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954f49f-20a8-4677-9133-92726057acd1 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:21.557649 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954f49f-20a8-4677-9133-92726057acd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:21.562316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:21.576405 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-d954f49f-20a8-4677-9133-92726057acd1 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:21.577595 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d954f49f-20a8-4677-9133-92726057acd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:21.597451 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:21.650414 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 276/564] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:21.651662 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954f49f-20a8-4677-9133-92726057acd1 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:21.664901 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f0677c-5b15-4a1d-bb94-a4338863e60b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:21.700802 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bde4bc-99c3-49d7-8fdd-84cdc42ef95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:21.701521 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bde4bc-99c3-49d7-8fdd-84cdc42ef95e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:21.701795 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bde4bc-99c3-49d7-8fdd-84cdc42ef95e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:21.702082 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bde4bc-99c3-49d7-8fdd-84cdc42ef95e None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:21.703103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bde4bc-99c3-49d7-8fdd-84cdc42ef95e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:21.703443 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bde4bc-99c3-49d7-8fdd-84cdc42ef95e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:21.704856 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bde4bc-99c3-49d7-8fdd-84cdc42ef95e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:21.716521 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 289/565] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:21.717342 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bde4bc-99c3-49d7-8fdd-84cdc42ef95e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:21.770893 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954f49f-20a8-4677-9133-92726057acd1 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:21.793461 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:21.809162 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b77098-0e2d-4c37-8da2-b050d341176f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:21.809998 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b77098-0e2d-4c37-8da2-b050d341176f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:21.810231 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b77098-0e2d-4c37-8da2-b050d341176f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:21.810507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b77098-0e2d-4c37-8da2-b050d341176f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:21.821188 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b77098-0e2d-4c37-8da2-b050d341176f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:21.822636 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b77098-0e2d-4c37-8da2-b050d341176f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:21.826796 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bde4bc-99c3-49d7-8fdd-84cdc42ef95e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:21.840551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-a7b77098-0e2d-4c37-8da2-b050d341176f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:21.840551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:21.840551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:21.840551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:21.840551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:21.840551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:21.840551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:21.840551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:21.840551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:21.840551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:21.840551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:21.840551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:21.840551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:21.840551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:21.840551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:21.841467 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:21.841467 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:21.841467 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:21.841467 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:21.841467 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:21.841467 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:21.841467 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:21.841467 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:21.841467 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:21.841467 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:21.841467 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:21.841467 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:21.841467 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:21.841467 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:21.841467 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:21.841467 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:21.841467 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:21.842356 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:21.842356 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:21.842356 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:21.842356 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:21.842356 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 277/566] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:21.842628 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b77098-0e2d-4c37-8da2-b050d341176f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:21.856534 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:21.868616 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b0d55-3839-4266-85a2-be51eb2a3e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:21.869295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b0d55-3839-4266-85a2-be51eb2a3e4f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:21.870965 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b0d55-3839-4266-85a2-be51eb2a3e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:21.870965 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104b0d55-3839-4266-85a2-be51eb2a3e4f None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:21.870965 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b0d55-3839-4266-85a2-be51eb2a3e4f None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:21.870965 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b0d55-3839-4266-85a2-be51eb2a3e4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:21.872286 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104b0d55-3839-4266-85a2-be51eb2a3e4f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:21.881658 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 290/567] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jun 30 10:14:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:21.886833 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b0d55-3839-4266-85a2-be51eb2a3e4f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:21.955005 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b77098-0e2d-4c37-8da2-b050d341176f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:21.990021 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b895f4-afae-453f-a59b-3bc040f6d538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:21.990622 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104b0d55-3839-4266-85a2-be51eb2a3e4f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:21.990816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b895f4-afae-453f-a59b-3bc040f6d538 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:21.991103 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b895f4-afae-453f-a59b-3bc040f6d538 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:21.991416 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b895f4-afae-453f-a59b-3bc040f6d538 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:22.001317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b895f4-afae-453f-a59b-3bc040f6d538 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:22.009701 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-59b895f4-afae-453f-a59b-3bc040f6d538 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:22.009701 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:22.009701 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:22.009701 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:22.009701 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:22.009701 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:22.009701 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:22.009701 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:22.009701 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:22.009701 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:22.009701 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:22.009701 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:22.009701 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:22.009701 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:22.009701 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:22.009701 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:22.010732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:22.010732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:22.010732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:22.010732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:22.010732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:22.010732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:22.010732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:22.010732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:22.010732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:22.010732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:22.010732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:22.010732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:22.010732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:22.010732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:22.010732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:22.010732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:22.011790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:22.011790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:22.011790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:22.011790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:22.011790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:22.011790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:22.011790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:22.011790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:22.011790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:22.011790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:22.011790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:22.011790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:22.011790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 278/568] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jun 30 10:14:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:22.011790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b895f4-afae-453f-a59b-3bc040f6d538 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:22.020094 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:22.025847 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5349b-5079-4994-92c3-f8e829eac96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:22.026643 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5349b-5079-4994-92c3-f8e829eac96a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:22.026929 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5349b-5079-4994-92c3-f8e829eac96a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:22.027205 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd5349b-5079-4994-92c3-f8e829eac96a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:22.034048 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5349b-5079-4994-92c3-f8e829eac96a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:22.035552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd5349b-5079-4994-92c3-f8e829eac96a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:22.050241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-cbd5349b-5079-4994-92c3-f8e829eac96a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:22.050241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:22.050241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:22.050241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:22.050241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:22.050241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:22.050241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:22.050241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:22.050241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:22.050241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:22.050241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:22.050241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:22.050241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:14:22.050241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:14:22.050241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:14:22.051120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:14:22.051120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:22.051120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:22.051120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:22.051120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:22.051120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:22.051120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:22.051120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:22.051120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:22.051120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:22.051120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:22.051120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:22.051120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:22.051120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:22.051120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:22.051120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:22.051120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:22.052226 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:14:22.052226 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:14:22.052226 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:14:22.052226 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:14:22.052226 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:14:22.052226 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:14:22.052226 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:22.052226 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:22.052226 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 291/569] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:22.052226 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5349b-5079-4994-92c3-f8e829eac96a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:22.078317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:22.113932 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b895f4-afae-453f-a59b-3bc040f6d538 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:22.147699 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122ab57-c2b0-4e5c-9448-9d75de158140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:22.148369 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122ab57-c2b0-4e5c-9448-9d75de158140 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:22.148656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122ab57-c2b0-4e5c-9448-9d75de158140 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:22.148936 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122ab57-c2b0-4e5c-9448-9d75de158140 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:22.149662 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3122ab57-c2b0-4e5c-9448-9d75de158140 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:22.149972 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3122ab57-c2b0-4e5c-9448-9d75de158140 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:22.151383 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3122ab57-c2b0-4e5c-9448-9d75de158140 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:22.152225 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd5349b-5079-4994-92c3-f8e829eac96a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:22.162984 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 279/570] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:22.164272 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdab45-05d2-482b-821b-42e878264429 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:22.164562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdab45-05d2-482b-821b-42e878264429 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:22.164842 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdab45-05d2-482b-821b-42e878264429 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:22.166074 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 280/571] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:22.166769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bdab45-05d2-482b-821b-42e878264429 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:22.193443 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0dd3c-f157-4c82-83aa-66c7ad5f7f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:22.194235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0dd3c-f157-4c82-83aa-66c7ad5f7f33 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:22.194527 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0dd3c-f157-4c82-83aa-66c7ad5f7f33 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:22.194800 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f0dd3c-f157-4c82-83aa-66c7ad5f7f33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:22.204927 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f0dd3c-f157-4c82-83aa-66c7ad5f7f33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:22.206397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f0dd3c-f157-4c82-83aa-66c7ad5f7f33 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:22.223494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-82f0dd3c-f157-4c82-83aa-66c7ad5f7f33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:22.223494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:22.223494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:22.223494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:22.223494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:22.223494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:22.223494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:22.223494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:22.223494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:22.223494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:22.223494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:22.223494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:22.223494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:22.223494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:22.223494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:22.224463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:22.224463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:22.224463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:22.224463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:22.224463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:22.224463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:22.224463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:22.224463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:22.224463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:22.224463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:22.224463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:22.224463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:22.224463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:22.224463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:22.224463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:22.224463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:22.224463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:22.225545 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:22.225545 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:22.225545 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:22.225545 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:22.225545 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 292/572] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:22.225545 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0dd3c-f157-4c82-83aa-66c7ad5f7f33 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:22.271385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:22.272496 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bdab45-05d2-482b-821b-42e878264429 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:22.297573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:22.306703 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9709f-ce20-401e-962b-fe411c998002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:22.307352 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9709f-ce20-401e-962b-fe411c998002 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:22.307640 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9709f-ce20-401e-962b-fe411c998002 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:22.307926 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9709f-ce20-401e-962b-fe411c998002 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:22.308668 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9709f-ce20-401e-962b-fe411c998002 None admin] RBAC: Adding query filter params (name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:22.308978 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9709f-ce20-401e-962b-fe411c998002 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:22.310348 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9709f-ce20-401e-962b-fe411c998002 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:22.318696 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 281/573] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jun 30 10:14:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:22.319520 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9709f-ce20-401e-962b-fe411c998002 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:22.324660 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f0dd3c-f157-4c82-83aa-66c7ad5f7f33 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:22.357586 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a087d4fd-85b2-4071-9c96-028d7710cad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:22.358247 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087d4fd-85b2-4071-9c96-028d7710cad9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:22.358533 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087d4fd-85b2-4071-9c96-028d7710cad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:22.358813 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a087d4fd-85b2-4071-9c96-028d7710cad9 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:22.359646 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a087d4fd-85b2-4071-9c96-028d7710cad9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:22.359961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a087d4fd-85b2-4071-9c96-028d7710cad9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:22.361484 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a087d4fd-85b2-4071-9c96-028d7710cad9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:22.374010 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 293/574] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:14:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:22.375208 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ec178-0799-4407-88bb-ffd0f1011e6b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:22.375527 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ec178-0799-4407-88bb-ffd0f1011e6b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:22.375808 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6ec178-0799-4407-88bb-ffd0f1011e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:22.393251 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ca6ec178-0799-4407-88bb-ffd0f1011e6b None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:22.394452 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ca6ec178-0799-4407-88bb-ffd0f1011e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:22.421706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9709f-ce20-401e-962b-fe411c998002 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:22.454144 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c317305-657b-4fe4-a457-b3c1a35dc335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:22.454934 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c317305-657b-4fe4-a457-b3c1a35dc335 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:22.455253 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c317305-657b-4fe4-a457-b3c1a35dc335 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:22.455553 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c317305-657b-4fe4-a457-b3c1a35dc335 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:22.460243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 294/575] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:22.461072 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ec178-0799-4407-88bb-ffd0f1011e6b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:22.462265 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c317305-657b-4fe4-a457-b3c1a35dc335 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:22.463758 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c317305-657b-4fe4-a457-b3c1a35dc335 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:22.478772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-2c317305-657b-4fe4-a457-b3c1a35dc335 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:22.478772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:22.478772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:22.478772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:22.478772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:22.478772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:22.478772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:22.478772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:22.478772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:22.478772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:22.478772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:22.478772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:22.478772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:14:22.478772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:14:22.478772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:14:22.479740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:14:22.479740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:22.479740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:22.479740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:22.479740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:22.479740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:22.479740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:22.479740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:22.479740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:22.479740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:22.479740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:22.479740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:22.479740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:22.479740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:22.479740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:22.479740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:22.479740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:22.480649 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:14:22.480649 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:14:22.480649 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:14:22.480649 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:14:22.480649 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:14:22.480649 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:14:22.480649 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:22.480649 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:22.480649 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 282/576] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:22.481133 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c317305-657b-4fe4-a457-b3c1a35dc335 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:22.499397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:22.534412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:22.566575 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6ec178-0799-4407-88bb-ffd0f1011e6b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:22.587675 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c317305-657b-4fe4-a457-b3c1a35dc335 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:22.600074 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce0cd5-a120-4625-b483-5c4aed86d1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:22.600780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce0cd5-a120-4625-b483-5c4aed86d1bc None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:22.601018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce0cd5-a120-4625-b483-5c4aed86d1bc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:22.601294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ce0cd5-a120-4625-b483-5c4aed86d1bc None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:22.602027 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce0cd5-a120-4625-b483-5c4aed86d1bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:22.602334 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce0cd5-a120-4625-b483-5c4aed86d1bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:22.603761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ce0cd5-a120-4625-b483-5c4aed86d1bc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:22.615148 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 295/577] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:22.615962 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce0cd5-a120-4625-b483-5c4aed86d1bc None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:22.622184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77cc773-5b06-4d79-9d63-0d0c39b22025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:22.622984 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cc773-5b06-4d79-9d63-0d0c39b22025 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:22.623296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cc773-5b06-4d79-9d63-0d0c39b22025 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:22.623582 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77cc773-5b06-4d79-9d63-0d0c39b22025 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:22.633973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77cc773-5b06-4d79-9d63-0d0c39b22025 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:22.635438 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77cc773-5b06-4d79-9d63-0d0c39b22025 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:22.652998 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-a77cc773-5b06-4d79-9d63-0d0c39b22025 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:22.652998 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:22.652998 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:22.652998 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:22.652998 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:22.652998 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:22.652998 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:22.652998 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:22.652998 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:22.652998 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:22.652998 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:22.652998 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:22.652998 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:22.652998 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:22.652998 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:22.654054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:22.654054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:22.654054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:22.654054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:22.654054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:22.654054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:22.654054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:22.654054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:22.654054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:22.654054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:22.654054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:22.654054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:22.654054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:22.654054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:22.654054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:22.654054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:22.654054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:22.654964 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:22.654964 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:22.654964 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:22.654964 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:22.654964 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 283/578] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:22.654964 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77cc773-5b06-4d79-9d63-0d0c39b22025 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:22.712756 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:22.719414 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ce0cd5-a120-4625-b483-5c4aed86d1bc None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:22.751594 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:22.752885 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49329d-5a64-4b87-994e-20d8f58941b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:22.753632 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49329d-5a64-4b87-994e-20d8f58941b4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:22.753986 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49329d-5a64-4b87-994e-20d8f58941b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:22.754332 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc49329d-5a64-4b87-994e-20d8f58941b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:22.754611 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77cc773-5b06-4d79-9d63-0d0c39b22025 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:22.755316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc49329d-5a64-4b87-994e-20d8f58941b4 None admin] RBAC: Adding query filter params (role.id=99d8cd9cd87e472290bf0de2710b5857, scope.project.id=50902e0539434c82b505dc5009aeaf85, user.id=07de926cb8594b098f79111dab8a9cc6) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:22.755702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc49329d-5a64-4b87-994e-20d8f58941b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:22.757177 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc49329d-5a64-4b87-994e-20d8f58941b4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:22.762554 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 296/579] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:14:22 2022] GET /identity/v3/role_assignments?user.id=07de926cb8594b098f79111dab8a9cc6&scope.project.id=50902e0539434c82b505dc5009aeaf85&role.id=99d8cd9cd87e472290bf0de2710b5857 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:22.763454 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49329d-5a64-4b87-994e-20d8f58941b4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:22.798220 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa26da-8a36-4e2d-85d5-f3fe36d2571d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:22.799582 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa26da-8a36-4e2d-85d5-f3fe36d2571d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:22.799891 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa26da-8a36-4e2d-85d5-f3fe36d2571d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:22.800318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa26da-8a36-4e2d-85d5-f3fe36d2571d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:22.812059 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa26da-8a36-4e2d-85d5-f3fe36d2571d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:22.820933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-04fa26da-8a36-4e2d-85d5-f3fe36d2571d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:22.820933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:22.820933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:22.820933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:22.820933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:22.820933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:22.820933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:22.820933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:22.820933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:22.820933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:22.820933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:22.820933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:22.820933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:22.820933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:22.820933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:22.820933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:22.821917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:22.821917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:22.821917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:22.821917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:22.821917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:22.821917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:22.821917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:22.821917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:22.821917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:22.821917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:22.821917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:22.821917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:22.821917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:22.821917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:22.821917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:22.821917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:22.822788 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:22.822788 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:22.822788 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:22.822788 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:22.822788 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:22.822788 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:22.822788 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:22.822788 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:22.822788 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:22.822788 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:22.822788 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:22.822788 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:22.822788 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 284/580] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jun 30 10:14:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:22.823528 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa26da-8a36-4e2d-85d5-f3fe36d2571d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:22.888647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc49329d-5a64-4b87-994e-20d8f58941b4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:22.934482 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc7a8d-921b-4f6b-8431-65c06e372ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:22.935162 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc7a8d-921b-4f6b-8431-65c06e372ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:22.935472 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc7a8d-921b-4f6b-8431-65c06e372ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:22.935752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fc7a8d-921b-4f6b-8431-65c06e372ff4 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:22.936575 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc7a8d-921b-4f6b-8431-65c06e372ff4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:22.936890 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc7a8d-921b-4f6b-8431-65c06e372ff4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:22.938330 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fc7a8d-921b-4f6b-8431-65c06e372ff4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:22.938949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa26da-8a36-4e2d-85d5-f3fe36d2571d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:22.956314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 297/581] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:14:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:22.957209 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc7a8d-921b-4f6b-8431-65c06e372ff4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:22.974836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811231f9-f609-4f0a-b8f6-06ae3d923306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:22.975666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811231f9-f609-4f0a-b8f6-06ae3d923306 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:22.975949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811231f9-f609-4f0a-b8f6-06ae3d923306 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:22.976226 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811231f9-f609-4f0a-b8f6-06ae3d923306 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:22.976681 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:22.984576 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811231f9-f609-4f0a-b8f6-06ae3d923306 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:22.990150 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811231f9-f609-4f0a-b8f6-06ae3d923306 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:23.001038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-811231f9-f609-4f0a-b8f6-06ae3d923306 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 10:14:23.001038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:23.001038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:23.001038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:23.001038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:23.001038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:23.001038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:23.001038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:23.001038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:23.001038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:23.001038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:23.001038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:23.001038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:14:23.001038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:14:23.001038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:14:23.001038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:14:23.002031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:23.002031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:23.002031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:14:23.002031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:23.002031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:14:23.002031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:23.002031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:23.002031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:23.002031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:23.002031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:23.002031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:23.002031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:23.002031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:23.002031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:23.002031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:23.002031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:23.002031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:23.002031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:23.003545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:23.003545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:23.003545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:14:23.003545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:14:23.003545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:14:23.003545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:14:23.003545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:14:23.003545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:14:23.003545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 10:14:23.003545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:23.003545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 285/582] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:14:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:23.004624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811231f9-f609-4f0a-b8f6-06ae3d923306 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:23.031070 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:23.072477 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fc7a8d-921b-4f6b-8431-65c06e372ff4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:23.111219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c1761-edbe-408c-8755-e0d3d2821692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:23.112076 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c1761-edbe-408c-8755-e0d3d2821692 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:23.112333 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c1761-edbe-408c-8755-e0d3d2821692 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:23.112525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c1761-edbe-408c-8755-e0d3d2821692 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:23.113240 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c1761-edbe-408c-8755-e0d3d2821692 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:23.113547 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c1761-edbe-408c-8755-e0d3d2821692 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:23.114912 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c1761-edbe-408c-8755-e0d3d2821692 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:23.117343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811231f9-f609-4f0a-b8f6-06ae3d923306 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:23.127557 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 298/583] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:23.131497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c1761-edbe-408c-8755-e0d3d2821692 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:23.158275 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e7e5f-0ac4-40d6-8dbb-bb8790be4d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:23.159060 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e7e5f-0ac4-40d6-8dbb-bb8790be4d97 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:23.159343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e7e5f-0ac4-40d6-8dbb-bb8790be4d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:23.159628 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1e7e5f-0ac4-40d6-8dbb-bb8790be4d97 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:23.160378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e7e5f-0ac4-40d6-8dbb-bb8790be4d97 None admin] RBAC: Adding query filter params (name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:23.160688 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e7e5f-0ac4-40d6-8dbb-bb8790be4d97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:23.162080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1e7e5f-0ac4-40d6-8dbb-bb8790be4d97 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:23.171114 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 286/584] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jun 30 10:14:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:23.171964 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e7e5f-0ac4-40d6-8dbb-bb8790be4d97 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:23.238729 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:23.260552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c1761-edbe-408c-8755-e0d3d2821692 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:23.277349 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:23.295709 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1e7e5f-0ac4-40d6-8dbb-bb8790be4d97 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:23.299309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ea697-07c6-43d4-987d-203e99582083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:23.300210 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ea697-07c6-43d4-987d-203e99582083 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:23.300585 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ea697-07c6-43d4-987d-203e99582083 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:23.300871 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ea697-07c6-43d4-987d-203e99582083 None admin] PATH_INFO: `/v3/projects/50902e0539434c82b505dc5009aeaf85/users/71f6d7715e6f49ffb9fdf6e1307bbd51/roles/99d8cd9cd87e472290bf0de2710b5857` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:23.319142 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3ea697-07c6-43d4-987d-203e99582083 None admin] RBAC: Authorizing `identity:create_grant(project_id=50902e0539434c82b505dc5009aeaf85, user_id=71f6d7715e6f49ffb9fdf6e1307bbd51, role_id=99d8cd9cd87e472290bf0de2710b5857)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:23.320650 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3ea697-07c6-43d4-987d-203e99582083 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:23.333266 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a806a50-2b4d-4535-92dc-d0c0ecbdfcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:23.333954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a806a50-2b4d-4535-92dc-d0c0ecbdfcdd None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:23.334240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a806a50-2b4d-4535-92dc-d0c0ecbdfcdd None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:23.334518 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a806a50-2b4d-4535-92dc-d0c0ecbdfcdd None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:23.335378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a806a50-2b4d-4535-92dc-d0c0ecbdfcdd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:23.335689 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a806a50-2b4d-4535-92dc-d0c0ecbdfcdd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:23.337134 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a806a50-2b4d-4535-92dc-d0c0ecbdfcdd None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:23.354712 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 287/585] 10.222.0.47 () {56 vars in 1137 bytes} [Thu Jun 30 10:14:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:23.355533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a806a50-2b4d-4535-92dc-d0c0ecbdfcdd None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:23.395308 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 299/586] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jun 30 10:14:23 2022] PUT /identity/v3/projects/50902e0539434c82b505dc5009aeaf85/users/71f6d7715e6f49ffb9fdf6e1307bbd51/roles/99d8cd9cd87e472290bf0de2710b5857 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:14:23.395308 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ea697-07c6-43d4-987d-203e99582083 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:23.485579 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a806a50-2b4d-4535-92dc-d0c0ecbdfcdd None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:23.531831 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ea697-07c6-43d4-987d-203e99582083 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:23.537591 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd853c0d-0e55-4cb0-be6b-75a195bc83c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:23.538739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd853c0d-0e55-4cb0-be6b-75a195bc83c6 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:23.539171 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd853c0d-0e55-4cb0-be6b-75a195bc83c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:23.539611 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd853c0d-0e55-4cb0-be6b-75a195bc83c6 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:23.550757 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd853c0d-0e55-4cb0-be6b-75a195bc83c6 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:23.552941 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd853c0d-0e55-4cb0-be6b-75a195bc83c6 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:23.556792 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:23.566388 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:23.573672 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-bd853c0d-0e55-4cb0-be6b-75a195bc83c6 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 10:14:23.573672 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:23.573672 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:23.573672 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:23.573672 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:23.573672 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:23.573672 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:23.573672 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:23.573672 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:23.573672 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:23.573672 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:23.573672 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:23.573672 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:14:23.573672 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:14:23.573672 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:14:23.573672 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:14:23.574617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:23.574617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:23.574617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:14:23.574617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:23.574617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:14:23.574617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:23.574617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:23.574617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:23.574617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:23.574617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:23.574617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:23.574617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:23.574617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:23.574617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:23.574617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:23.574617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:23.574617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:23.574617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:23.575586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:23.575586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:23.575586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:14:23.575586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:14:23.575586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:14:23.575586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:14:23.575586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:14:23.575586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:14:23.575586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 10:14:23.575586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:23.575586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 288/587] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:14:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:23.576188 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd853c0d-0e55-4cb0-be6b-75a195bc83c6 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:23.581330 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60275c55-5314-46a8-9f7a-4828154c5051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:23.582230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60275c55-5314-46a8-9f7a-4828154c5051 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:23.582520 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60275c55-5314-46a8-9f7a-4828154c5051 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:23.582758 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60275c55-5314-46a8-9f7a-4828154c5051 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:23.601559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60275c55-5314-46a8-9f7a-4828154c5051 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:23.601559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60275c55-5314-46a8-9f7a-4828154c5051 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:23.624176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-60275c55-5314-46a8-9f7a-4828154c5051 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:23.624176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:23.624176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:23.624176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:23.624176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:23.624176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:23.624176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:23.624176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:23.624176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:23.624176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:23.624176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:23.624176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:23.624176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:23.624176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:23.624176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:23.632055 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:23.632055 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:23.632055 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:23.632055 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:23.632055 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:23.632055 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:23.632055 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:23.632055 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:23.632055 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:23.632055 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:23.632055 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:23.632055 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:23.632055 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:23.632055 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:23.632055 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:23.632055 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:23.632055 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:23.632955 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:23.632955 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:23.632955 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:23.632955 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:23.632955 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 300/588] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:23.632955 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60275c55-5314-46a8-9f7a-4828154c5051 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:23.698965 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd853c0d-0e55-4cb0-be6b-75a195bc83c6 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:23.737332 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dba890-9854-4bb5-90a6-e0a9b1737bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:23.738051 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dba890-9854-4bb5-90a6-e0a9b1737bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:23.738333 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dba890-9854-4bb5-90a6-e0a9b1737bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:23.738608 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dba890-9854-4bb5-90a6-e0a9b1737bf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:23.749404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dba890-9854-4bb5-90a6-e0a9b1737bf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:23.750627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dba890-9854-4bb5-90a6-e0a9b1737bf9 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:23.764702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60275c55-5314-46a8-9f7a-4828154c5051 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:23.769370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-20dba890-9854-4bb5-90a6-e0a9b1737bf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:23.769370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:23.769370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:23.769370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:23.769370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:23.769370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:23.769370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:23.769370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:23.769370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:23.769370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:23.769370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:23.769370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:23.769370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:23.769370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:23.769370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:23.770249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:23.770249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:23.770249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:23.770249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:23.770249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:23.770249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:23.770249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:23.770249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:23.770249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:23.770249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:23.770249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:23.770249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:23.770249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:23.770249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:23.770249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:23.770249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:23.770249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:23.772179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:23.772179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:23.772179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:23.772179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:23.772179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 289/589] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:23.772179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dba890-9854-4bb5-90a6-e0a9b1737bf9 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:23.809823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db311bbb-1a0b-423b-9165-dd6b5962ba8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:23.810506 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db311bbb-1a0b-423b-9165-dd6b5962ba8b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:23.810788 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db311bbb-1a0b-423b-9165-dd6b5962ba8b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:23.811116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db311bbb-1a0b-423b-9165-dd6b5962ba8b None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:23.811983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db311bbb-1a0b-423b-9165-dd6b5962ba8b None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:23.812339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db311bbb-1a0b-423b-9165-dd6b5962ba8b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:23.813852 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db311bbb-1a0b-423b-9165-dd6b5962ba8b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:23.833694 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 301/590] 10.222.0.47 () {56 vars in 1137 bytes} [Thu Jun 30 10:14:23 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:23.834466 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db311bbb-1a0b-423b-9165-dd6b5962ba8b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:23.839642 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:23.882527 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:23.911201 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dba890-9854-4bb5-90a6-e0a9b1737bf9 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:23.948076 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db311bbb-1a0b-423b-9165-dd6b5962ba8b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:23.956366 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf87c4e-e750-48fe-896d-372d320ee02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:23.957357 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf87c4e-e750-48fe-896d-372d320ee02e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:23.957735 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf87c4e-e750-48fe-896d-372d320ee02e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:23.958107 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf87c4e-e750-48fe-896d-372d320ee02e None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:23.959254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf87c4e-e750-48fe-896d-372d320ee02e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:23.959254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf87c4e-e750-48fe-896d-372d320ee02e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:23.960940 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf87c4e-e750-48fe-896d-372d320ee02e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:23.978243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 290/591] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:23.978243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf87c4e-e750-48fe-896d-372d320ee02e None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:23.999812 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bec400-cc31-413c-ae66-64e8de61f71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:24.001204 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bec400-cc31-413c-ae66-64e8de61f71c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:24.001934 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bec400-cc31-413c-ae66-64e8de61f71c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:24.002363 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bec400-cc31-413c-ae66-64e8de61f71c None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:24.003237 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bec400-cc31-413c-ae66-64e8de61f71c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:24.003987 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bec400-cc31-413c-ae66-64e8de61f71c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:24.006196 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bec400-cc31-413c-ae66-64e8de61f71c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:24.020696 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 302/592] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:24.022105 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bec400-cc31-413c-ae66-64e8de61f71c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:24.086285 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf87c4e-e750-48fe-896d-372d320ee02e None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:24.101193 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:24.123826 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f5fe1-5487-403a-9e29-0572dde7b32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:24.124208 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:24.124905 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f5fe1-5487-403a-9e29-0572dde7b32a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:24.125256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f5fe1-5487-403a-9e29-0572dde7b32a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:24.125629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43f5fe1-5487-403a-9e29-0572dde7b32a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:24.129201 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bec400-cc31-413c-ae66-64e8de61f71c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:24.137546 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43f5fe1-5487-403a-9e29-0572dde7b32a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:24.139123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43f5fe1-5487-403a-9e29-0572dde7b32a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:24.157979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-f43f5fe1-5487-403a-9e29-0572dde7b32a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:24.157979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:24.157979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:24.157979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:24.157979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:24.157979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:24.157979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:24.157979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:24.157979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:24.157979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:24.157979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:24.157979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:24.157979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:24.157979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:24.157979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:24.159555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:24.159555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:24.159555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:24.159555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:24.159555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:24.159555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:24.159555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:24.159555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:24.159555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:24.159555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:24.159555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:24.159555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:24.159555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:24.159555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:24.159555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:24.159555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:24.159555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:24.160634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:24.160634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:24.160634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:24.160634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:24.160634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 291/593] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:24.161741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f5fe1-5487-403a-9e29-0572dde7b32a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:24.169706 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df68317-8f56-419a-8ee0-036d0bf6a77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:24.170885 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df68317-8f56-419a-8ee0-036d0bf6a77d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:24.174692 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df68317-8f56-419a-8ee0-036d0bf6a77d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:24.174692 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df68317-8f56-419a-8ee0-036d0bf6a77d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:24.181225 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df68317-8f56-419a-8ee0-036d0bf6a77d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:24.182882 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df68317-8f56-419a-8ee0-036d0bf6a77d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:24.199913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-7df68317-8f56-419a-8ee0-036d0bf6a77d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 10:14:24.199913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:24.199913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:24.199913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:24.199913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:24.199913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:24.199913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:24.199913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:24.199913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:24.199913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:24.199913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:24.199913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:24.199913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:14:24.199913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:14:24.199913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:14:24.199913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:14:24.200881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:24.200881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:24.200881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:14:24.200881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:24.200881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:14:24.200881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:24.200881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:24.200881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:24.200881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:24.200881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:24.200881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:24.200881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:24.200881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:24.200881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:24.200881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:24.200881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:24.200881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:24.200881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:24.201829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:24.201829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:24.201829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:14:24.201829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:14:24.201829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:14:24.201829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:14:24.201829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:14:24.201829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:14:24.201829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 10:14:24.201829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:24.209501 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 303/594] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jun 30 10:14:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:24.210755 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df68317-8f56-419a-8ee0-036d0bf6a77d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:24.272586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43f5fe1-5487-403a-9e29-0572dde7b32a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:24.309250 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6105a53-e258-43b6-99fc-d687d9c9ee90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:24.310139 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6105a53-e258-43b6-99fc-d687d9c9ee90 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:24.310488 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6105a53-e258-43b6-99fc-d687d9c9ee90 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:24.310830 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6105a53-e258-43b6-99fc-d687d9c9ee90 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:24.318258 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6105a53-e258-43b6-99fc-d687d9c9ee90 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:24.319855 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6105a53-e258-43b6-99fc-d687d9c9ee90 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:24.320644 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df68317-8f56-419a-8ee0-036d0bf6a77d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:24.334905 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-c6105a53-e258-43b6-99fc-d687d9c9ee90 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:24.334905 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:24.334905 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:24.334905 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:24.334905 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:24.334905 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:24.334905 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:24.334905 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:24.334905 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:24.334905 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:24.334905 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:24.334905 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:24.334905 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:14:24.334905 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:14:24.334905 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:14:24.335869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:14:24.335869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:24.335869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:24.335869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:24.335869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:24.335869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:24.335869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:24.335869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:24.335869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:24.335869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:24.335869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:24.335869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:24.335869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:24.335869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:24.335869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:24.335869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:24.335869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:24.337243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:14:24.337243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:14:24.337243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:14:24.337243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:14:24.337243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:14:24.337243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:14:24.337243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:24.337243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:24.337243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 292/595] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:24.337755 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6105a53-e258-43b6-99fc-d687d9c9ee90 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:24.361682 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ab5e4-0caa-4cb3-9238-fc68aad0bde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:24.362686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ab5e4-0caa-4cb3-9238-fc68aad0bde2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:24.363164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ab5e4-0caa-4cb3-9238-fc68aad0bde2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:24.363648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ab5e4-0caa-4cb3-9238-fc68aad0bde2 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:24.364651 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ab5e4-0caa-4cb3-9238-fc68aad0bde2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:24.365148 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ab5e4-0caa-4cb3-9238-fc68aad0bde2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:24.365580 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:24.367884 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ab5e4-0caa-4cb3-9238-fc68aad0bde2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:24.380694 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 304/596] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:24.381876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ab5e4-0caa-4cb3-9238-fc68aad0bde2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:24.404483 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:24.447455 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6105a53-e258-43b6-99fc-d687d9c9ee90 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:24.482743 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1c345-2495-475b-80c6-502c1ba192e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:24.483504 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1c345-2495-475b-80c6-502c1ba192e9 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:24.483859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1c345-2495-475b-80c6-502c1ba192e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:24.484204 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e1c345-2495-475b-80c6-502c1ba192e9 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:24.485313 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1c345-2495-475b-80c6-502c1ba192e9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:24.485699 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1c345-2495-475b-80c6-502c1ba192e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:24.490938 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e1c345-2495-475b-80c6-502c1ba192e9 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:24.496026 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ab5e4-0caa-4cb3-9238-fc68aad0bde2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:24.518109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 293/597] 10.222.0.47 () {56 vars in 1139 bytes} [Thu Jun 30 10:14:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:24.519344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1c345-2495-475b-80c6-502c1ba192e9 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:24.543143 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c9497-7775-40bb-9125-e53f5e42061e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:24.544186 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c9497-7775-40bb-9125-e53f5e42061e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:24.544647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c9497-7775-40bb-9125-e53f5e42061e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:24.545647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c9497-7775-40bb-9125-e53f5e42061e None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:24.546924 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c9497-7775-40bb-9125-e53f5e42061e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:24.547439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c9497-7775-40bb-9125-e53f5e42061e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:24.550421 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c9497-7775-40bb-9125-e53f5e42061e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:24.567569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 305/598] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:14:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:24.569146 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c9497-7775-40bb-9125-e53f5e42061e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:24.616911 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:24.639436 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e1c345-2495-475b-80c6-502c1ba192e9 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:24.653870 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:24.677761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c9497-7775-40bb-9125-e53f5e42061e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:24.680715 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a76eaa-f450-4f7f-832a-5fdcb6bde9e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:24.681761 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a76eaa-f450-4f7f-832a-5fdcb6bde9e0 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:24.682283 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a76eaa-f450-4f7f-832a-5fdcb6bde9e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:24.682708 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a76eaa-f450-4f7f-832a-5fdcb6bde9e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:24.691665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a76eaa-f450-4f7f-832a-5fdcb6bde9e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:24.693640 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a76eaa-f450-4f7f-832a-5fdcb6bde9e0 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:24.709326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-60a76eaa-f450-4f7f-832a-5fdcb6bde9e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:24.709326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:24.709326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:24.709326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:24.709326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:24.709326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:24.709326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:24.709326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:24.709326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:24.709326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:24.709326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:24.709326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:24.709326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:14:24.709326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:14:24.709326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:14:24.710462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:14:24.710462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:24.710462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:24.710462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:24.710462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:24.710462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:24.710462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:24.710462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:24.710462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:24.710462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:24.710462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:24.710462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:24.710462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:24.710462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:24.710462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:24.710462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:24.710462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:24.711610 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:14:24.711610 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:14:24.711610 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:14:24.711610 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:14:24.711610 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:14:24.711610 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:14:24.711610 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:24.711610 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:24.713493 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 294/599] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:24.714444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a76eaa-f450-4f7f-832a-5fdcb6bde9e0 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:24.719831 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948ecd4-2a80-4cde-8d60-d31c8c44eb70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:24.720570 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948ecd4-2a80-4cde-8d60-d31c8c44eb70 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:24.721391 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948ecd4-2a80-4cde-8d60-d31c8c44eb70 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:24.721666 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948ecd4-2a80-4cde-8d60-d31c8c44eb70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:24.733031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948ecd4-2a80-4cde-8d60-d31c8c44eb70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:24.734554 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948ecd4-2a80-4cde-8d60-d31c8c44eb70 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:24.757260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-3948ecd4-2a80-4cde-8d60-d31c8c44eb70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:24.757260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:24.757260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:24.757260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:24.757260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:24.757260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:24.757260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:24.757260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:24.757260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:24.757260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:24.757260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:24.757260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:24.757260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:24.757260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:24.757260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:24.758185 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:24.758185 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:24.758185 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:24.758185 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:24.758185 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:24.758185 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:24.758185 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:24.758185 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:24.758185 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:24.758185 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:24.758185 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:24.758185 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:24.758185 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:24.758185 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:24.758185 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:24.758185 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:24.758185 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:24.759075 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:24.759075 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:24.759075 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:24.759075 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:24.759075 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 306/600] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:24.759868 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948ecd4-2a80-4cde-8d60-d31c8c44eb70 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:24.832827 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a76eaa-f450-4f7f-832a-5fdcb6bde9e0 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:24.869909 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2708d234-6876-4b6d-ba78-9178897ca736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:24.870896 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708d234-6876-4b6d-ba78-9178897ca736 None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:24.871185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708d234-6876-4b6d-ba78-9178897ca736 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:24.871659 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708d234-6876-4b6d-ba78-9178897ca736 None admin] PATH_INFO: `/v3/projects/50902e0539434c82b505dc5009aeaf85/users/c7a61d85908d42698b27b9fc00daa832/roles/99d8cd9cd87e472290bf0de2710b5857` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:24.878098 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948ecd4-2a80-4cde-8d60-d31c8c44eb70 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:24.884954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:24.889085 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2708d234-6876-4b6d-ba78-9178897ca736 None admin] RBAC: Authorizing `identity:create_grant(project_id=50902e0539434c82b505dc5009aeaf85, user_id=c7a61d85908d42698b27b9fc00daa832, role_id=99d8cd9cd87e472290bf0de2710b5857)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:24.890613 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2708d234-6876-4b6d-ba78-9178897ca736 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:24.906585 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:24.912715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1601c-66d7-41b8-852b-3c554ca50a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:24.913372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1601c-66d7-41b8-852b-3c554ca50a73 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:24.913657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1601c-66d7-41b8-852b-3c554ca50a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:24.913961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a1601c-66d7-41b8-852b-3c554ca50a73 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:24.914772 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1601c-66d7-41b8-852b-3c554ca50a73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:24.915082 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1601c-66d7-41b8-852b-3c554ca50a73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:24.916537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a1601c-66d7-41b8-852b-3c554ca50a73 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:24.928799 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 307/601] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:14:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:24.929528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1601c-66d7-41b8-852b-3c554ca50a73 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:24.954954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 295/602] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jun 30 10:14:24 2022] PUT /identity/v3/projects/50902e0539434c82b505dc5009aeaf85/users/c7a61d85908d42698b27b9fc00daa832/roles/99d8cd9cd87e472290bf0de2710b5857 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:14:24.965207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a3be2-0915-4e53-af75-79bc6e2ca9f2 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:24.965498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a3be2-0915-4e53-af75-79bc6e2ca9f2 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:24.965805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a3be2-0915-4e53-af75-79bc6e2ca9f2 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:24.971516 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 296/603] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:24.971516 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a3be2-0915-4e53-af75-79bc6e2ca9f2 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:25.051885 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a1601c-66d7-41b8-852b-3c554ca50a73 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:25.103844 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788bc92c-4e75-462f-ae6b-0c08935911e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:25.105308 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bc92c-4e75-462f-ae6b-0c08935911e0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:25.105698 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bc92c-4e75-462f-ae6b-0c08935911e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:25.106348 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788bc92c-4e75-462f-ae6b-0c08935911e0 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:25.107211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788bc92c-4e75-462f-ae6b-0c08935911e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:25.107651 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788bc92c-4e75-462f-ae6b-0c08935911e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:25.109952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788bc92c-4e75-462f-ae6b-0c08935911e0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:25.128228 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 308/604] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:25.132784 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab1cfb-750b-4087-87ed-a24a4aacf72f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:25.132784 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab1cfb-750b-4087-87ed-a24a4aacf72f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:25.132784 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ab1cfb-750b-4087-87ed-a24a4aacf72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:25.133323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a3be2-0915-4e53-af75-79bc6e2ca9f2 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:25.152020 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c7ab1cfb-750b-4087-87ed-a24a4aacf72f None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:25.153326 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c7ab1cfb-750b-4087-87ed-a24a4aacf72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:25.175587 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2144d-8cd0-4926-9ac0-fda39b467588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:25.176498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2144d-8cd0-4926-9ac0-fda39b467588 None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:25.176817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2144d-8cd0-4926-9ac0-fda39b467588 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:25.177099 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2144d-8cd0-4926-9ac0-fda39b467588 None admin] PATH_INFO: `/v3/projects/50902e0539434c82b505dc5009aeaf85/users/07ab12e5d1d94a009c4d414e5c43ac92/roles/99d8cd9cd87e472290bf0de2710b5857` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:25.193511 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2144d-8cd0-4926-9ac0-fda39b467588 None admin] RBAC: Authorizing `identity:create_grant(project_id=50902e0539434c82b505dc5009aeaf85, user_id=07ab12e5d1d94a009c4d414e5c43ac92, role_id=99d8cd9cd87e472290bf0de2710b5857)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:25.195027 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea2144d-8cd0-4926-9ac0-fda39b467588 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:25.218804 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:25.231242 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 309/605] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:25.232107 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab1cfb-750b-4087-87ed-a24a4aacf72f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:25.264042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 297/606] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jun 30 10:14:24 2022] PUT /identity/v3/projects/50902e0539434c82b505dc5009aeaf85/users/07ab12e5d1d94a009c4d414e5c43ac92/roles/99d8cd9cd87e472290bf0de2710b5857 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:14:25.268570 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2144d-8cd0-4926-9ac0-fda39b467588 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:25.375630 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:25.434192 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ab1cfb-750b-4087-87ed-a24a4aacf72f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:25.478239 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2144d-8cd0-4926-9ac0-fda39b467588 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:25.479848 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de7220-3572-476e-aa6a-2312b2467579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:25.480661 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de7220-3572-476e-aa6a-2312b2467579 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:25.487048 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de7220-3572-476e-aa6a-2312b2467579 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:25.487048 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76de7220-3572-476e-aa6a-2312b2467579 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:25.500334 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de7220-3572-476e-aa6a-2312b2467579 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:25.502401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76de7220-3572-476e-aa6a-2312b2467579 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:25.524241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-76de7220-3572-476e-aa6a-2312b2467579 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:25.524241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:25.524241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:25.524241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:25.524241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:25.524241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:25.524241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:25.524241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:25.524241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:25.524241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:25.524241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:25.524241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:25.524241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:14:25.524241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:14:25.524241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:14:25.525243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:14:25.525243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:25.525243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:25.525243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:25.525243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:25.525243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:25.525243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:25.525243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:25.525243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:25.525243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:25.525243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:25.525243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:25.525243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:25.525243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:25.525243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:25.525243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:25.525243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:25.526197 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:14:25.526197 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:14:25.526197 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:14:25.526197 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:14:25.526197 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:14:25.526197 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:14:25.526197 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:25.526197 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:25.526197 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 310/607] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:25.531205 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de7220-3572-476e-aa6a-2312b2467579 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:25.576725 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:25.600994 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19be841-8ee7-4690-b3b8-17e6237f6409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:25.601822 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19be841-8ee7-4690-b3b8-17e6237f6409 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:25.602113 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19be841-8ee7-4690-b3b8-17e6237f6409 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:25.602394 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19be841-8ee7-4690-b3b8-17e6237f6409 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:25.636208 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19be841-8ee7-4690-b3b8-17e6237f6409 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:25.638386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76de7220-3572-476e-aa6a-2312b2467579 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:25.644714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-c19be841-8ee7-4690-b3b8-17e6237f6409 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:25.644714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:25.644714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:25.644714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:25.644714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:25.644714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:25.644714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:25.644714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:25.644714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:25.644714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:25.644714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:25.644714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:25.644714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:25.644714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:25.644714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:25.644714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:25.645692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:25.645692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:25.645692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:25.645692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:25.645692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:25.645692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:25.645692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:25.645692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:25.645692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:25.645692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:25.645692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:25.645692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:25.645692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:25.645692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:25.645692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:25.645692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:25.646796 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:25.646796 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:25.646796 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:25.646796 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:25.646796 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:25.646796 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:25.646796 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:25.646796 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:25.646796 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:25.646796 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:25.646796 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:25.646796 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:25.651428 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 298/608] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jun 30 10:14:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:25.661280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19be841-8ee7-4690-b3b8-17e6237f6409 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:25.695809 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e78a7f-7f4d-498f-9d20-165b6568dce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:25.698373 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e78a7f-7f4d-498f-9d20-165b6568dce9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:25.698665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e78a7f-7f4d-498f-9d20-165b6568dce9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:25.698944 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e78a7f-7f4d-498f-9d20-165b6568dce9 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:25.699774 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e78a7f-7f4d-498f-9d20-165b6568dce9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:25.700087 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e78a7f-7f4d-498f-9d20-165b6568dce9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:25.701630 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e78a7f-7f4d-498f-9d20-165b6568dce9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:25.716121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 311/609] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:14:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:14:25.717518 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c730484-9cc3-460a-a9e0-ea720454fdab None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:25.717810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c730484-9cc3-460a-a9e0-ea720454fdab None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:25.718086 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c730484-9cc3-460a-a9e0-ea720454fdab None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:25.719562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 312/610] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:25.723637 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c730484-9cc3-460a-a9e0-ea720454fdab None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:25.796813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19be841-8ee7-4690-b3b8-17e6237f6409 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:25.802443 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:25.846219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adc4b5-d729-4f56-9863-bdc11bcc8af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:25.848229 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:25.848501 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adc4b5-d729-4f56-9863-bdc11bcc8af3 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:25.849929 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adc4b5-d729-4f56-9863-bdc11bcc8af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:25.850409 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adc4b5-d729-4f56-9863-bdc11bcc8af3 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:25.851475 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90adc4b5-d729-4f56-9863-bdc11bcc8af3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:25.852322 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c730484-9cc3-460a-a9e0-ea720454fdab None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:25.852448 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90adc4b5-d729-4f56-9863-bdc11bcc8af3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:25.854349 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90adc4b5-d729-4f56-9863-bdc11bcc8af3 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:25.866653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 299/611] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jun 30 10:14:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:25.868395 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b55525-9d1d-4c37-a8a2-ca3d05cff44a None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:25.868883 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b55525-9d1d-4c37-a8a2-ca3d05cff44a None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:25.869374 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b55525-9d1d-4c37-a8a2-ca3d05cff44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:25.890509 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-78b55525-9d1d-4c37-a8a2-ca3d05cff44a None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:25.892724 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-78b55525-9d1d-4c37-a8a2-ca3d05cff44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:25.905703 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0e221-d258-4b1b-a17d-301d4fe12d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:25.907538 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0e221-d258-4b1b-a17d-301d4fe12d2c None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:25.908112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0e221-d258-4b1b-a17d-301d4fe12d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:25.908637 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f0e221-d258-4b1b-a17d-301d4fe12d2c None admin] PATH_INFO: `/v3/projects/50902e0539434c82b505dc5009aeaf85/users/059b05007c0144f7a50dbd9f0ef41359/roles/99d8cd9cd87e472290bf0de2710b5857` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:25.929253 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f0e221-d258-4b1b-a17d-301d4fe12d2c None admin] RBAC: Authorizing `identity:create_grant(project_id=50902e0539434c82b505dc5009aeaf85, user_id=059b05007c0144f7a50dbd9f0ef41359, role_id=99d8cd9cd87e472290bf0de2710b5857)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:25.931535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f0e221-d258-4b1b-a17d-301d4fe12d2c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:25.967927 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 300/612] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:25.976139 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b55525-9d1d-4c37-a8a2-ca3d05cff44a None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:25.996968 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 313/613] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jun 30 10:14:25 2022] PUT /identity/v3/projects/50902e0539434c82b505dc5009aeaf85/users/059b05007c0144f7a50dbd9f0ef41359/roles/99d8cd9cd87e472290bf0de2710b5857 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:14:26.006172 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0e221-d258-4b1b-a17d-301d4fe12d2c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:26.169988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f0e221-d258-4b1b-a17d-301d4fe12d2c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:26.214018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb190514-dec8-468d-9f51-918d072e0ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:26.215299 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb190514-dec8-468d-9f51-918d072e0ffa None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:26.215299 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb190514-dec8-468d-9f51-918d072e0ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:26.215561 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb190514-dec8-468d-9f51-918d072e0ffa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:26.229157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb190514-dec8-468d-9f51-918d072e0ffa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:26.237259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-eb190514-dec8-468d-9f51-918d072e0ffa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:26.237259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:26.237259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:26.237259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:26.237259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:26.237259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:26.237259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:26.237259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:26.237259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:26.237259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:26.237259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:26.237259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:26.237259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:26.237259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:26.237259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:26.237259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:26.238230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:26.238230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:26.238230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:26.238230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:26.238230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:26.238230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:26.238230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:26.238230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:26.238230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:26.238230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:26.238230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:26.238230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:26.238230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:26.238230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:26.238230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:26.238230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:26.239355 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:26.239355 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:26.239355 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:26.239355 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:26.239355 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:26.239355 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:26.239355 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:26.239355 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:26.239355 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:26.239355 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:26.239355 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:26.239355 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:26.239355 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 314/614] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jun 30 10:14:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:26.245175 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb190514-dec8-468d-9f51-918d072e0ffa None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:26.246098 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:26.254924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b55525-9d1d-4c37-a8a2-ca3d05cff44a None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:26.264911 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:26.304868 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f821fb4-9a92-4031-94c7-bdfdf5bd881c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:26.305834 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f821fb4-9a92-4031-94c7-bdfdf5bd881c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:26.306493 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f821fb4-9a92-4031-94c7-bdfdf5bd881c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:26.306851 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f821fb4-9a92-4031-94c7-bdfdf5bd881c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:26.321301 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f821fb4-9a92-4031-94c7-bdfdf5bd881c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:26.323435 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f821fb4-9a92-4031-94c7-bdfdf5bd881c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:26.351606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-4f821fb4-9a92-4031-94c7-bdfdf5bd881c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:26.351606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:26.351606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:26.351606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:26.351606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:26.351606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:26.351606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:26.351606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:26.351606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:26.351606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:26.351606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:26.351606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:26.351606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:26.351606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:26.351606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:26.352567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:26.352567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:26.352567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:26.352567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:26.352567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:26.352567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:26.352567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:26.352567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:26.352567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:26.352567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:26.352567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:26.352567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:26.352567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:26.352567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:26.352567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:26.352567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:26.352567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:26.353443 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:26.353443 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:26.353443 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:26.353443 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:26.353443 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 301/615] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 385 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:26.362172 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f821fb4-9a92-4031-94c7-bdfdf5bd881c None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:26.399435 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb190514-dec8-468d-9f51-918d072e0ffa None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:26.473962 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f53091-eeec-4f0e-897b-ed5f0b61e0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:26.476816 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f53091-eeec-4f0e-897b-ed5f0b61e0a6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:26.477370 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f53091-eeec-4f0e-897b-ed5f0b61e0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:26.477858 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f53091-eeec-4f0e-897b-ed5f0b61e0a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:26.485215 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f53091-eeec-4f0e-897b-ed5f0b61e0a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:26.488422 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f53091-eeec-4f0e-897b-ed5f0b61e0a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:26.492366 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f53091-eeec-4f0e-897b-ed5f0b61e0a6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:26.515259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 315/616] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jun 30 10:14:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:14:26.520829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f53091-eeec-4f0e-897b-ed5f0b61e0a6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:26.566664 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f821fb4-9a92-4031-94c7-bdfdf5bd881c None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:26.607728 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a647c-d191-4155-bf32-2f8d604a487f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:26.610001 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a647c-d191-4155-bf32-2f8d604a487f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:26.610491 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a647c-d191-4155-bf32-2f8d604a487f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:26.611324 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a647c-d191-4155-bf32-2f8d604a487f None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:26.612660 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29a647c-d191-4155-bf32-2f8d604a487f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:26.614056 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29a647c-d191-4155-bf32-2f8d604a487f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:26.616471 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29a647c-d191-4155-bf32-2f8d604a487f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:26.639347 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 302/617] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:26.645706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a647c-d191-4155-bf32-2f8d604a487f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:26.655870 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:26.673911 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:26.720317 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f53091-eeec-4f0e-897b-ed5f0b61e0a6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:26.800364 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc867e1-9021-419d-b353-e8ca65f16c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:26.800952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc867e1-9021-419d-b353-e8ca65f16c4b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:26.801244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc867e1-9021-419d-b353-e8ca65f16c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:26.801524 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc867e1-9021-419d-b353-e8ca65f16c4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:26.816308 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a647c-d191-4155-bf32-2f8d604a487f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:26.816975 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc867e1-9021-419d-b353-e8ca65f16c4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:26.818451 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc867e1-9021-419d-b353-e8ca65f16c4b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:26.848556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-bfc867e1-9021-419d-b353-e8ca65f16c4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:26.848556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:26.848556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:26.848556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:26.848556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:26.848556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:26.848556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:26.848556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:26.848556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:26.848556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:26.848556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:26.848556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:26.848556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:26.848556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:26.848556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:26.849528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:26.849528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:26.849528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:26.849528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:26.849528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:26.849528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:26.849528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:26.849528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:26.849528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:26.849528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:26.849528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:26.849528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:26.849528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:26.849528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:26.849528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:26.849528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:26.849528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:26.851210 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:26.851210 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:26.851210 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:26.851210 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:26.851210 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 316/618] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:26.857993 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2dd81-f78b-4db1-9b2c-9da557bf1987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:26.859067 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2dd81-f78b-4db1-9b2c-9da557bf1987 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:26.859591 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2dd81-f78b-4db1-9b2c-9da557bf1987 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:26.860074 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb2dd81-f78b-4db1-9b2c-9da557bf1987 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:26.860196 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc867e1-9021-419d-b353-e8ca65f16c4b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:26.868770 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb2dd81-f78b-4db1-9b2c-9da557bf1987 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:26.870483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb2dd81-f78b-4db1-9b2c-9da557bf1987 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:26.886434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-bfb2dd81-f78b-4db1-9b2c-9da557bf1987 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 10:14:26.886434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:26.886434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:26.886434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:26.886434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:26.886434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:26.886434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:26.886434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:26.886434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:26.886434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:26.886434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:26.886434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:26.886434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:14:26.886434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:14:26.886434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:14:26.886434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:14:26.887547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:26.887547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:26.887547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:14:26.887547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:26.887547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:14:26.887547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:26.887547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:26.887547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:26.887547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:26.887547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:26.887547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:26.887547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:26.887547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:26.887547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:26.887547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:26.887547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:26.887547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:26.887547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:26.888513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:26.888513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:26.888513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:14:26.888513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:14:26.888513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:14:26.888513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:14:26.888513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:14:26.888513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:14:26.888513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 10:14:26.888513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:26.891320 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 303/619] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:14:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:14:26.904366 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2dd81-f78b-4db1-9b2c-9da557bf1987 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:26.975631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:27.011053 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb2dd81-f78b-4db1-9b2c-9da557bf1987 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:27.047526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b8fb6-b32d-4a91-be98-2c6bcaa0b8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:27.048216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b8fb6-b32d-4a91-be98-2c6bcaa0b8ce None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:27.048509 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b8fb6-b32d-4a91-be98-2c6bcaa0b8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:27.049333 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1b8fb6-b32d-4a91-be98-2c6bcaa0b8ce None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:27.050194 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1b8fb6-b32d-4a91-be98-2c6bcaa0b8ce None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:27.050567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1b8fb6-b32d-4a91-be98-2c6bcaa0b8ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:27.052330 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1b8fb6-b32d-4a91-be98-2c6bcaa0b8ce None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:27.070738 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 304/620] 10.222.0.47 () {56 vars in 1135 bytes} [Thu Jun 30 10:14:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:27.080294 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b8fb6-b32d-4a91-be98-2c6bcaa0b8ce None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:27.081789 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc867e1-9021-419d-b353-e8ca65f16c4b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:27.124301 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:27.155149 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde21d7-f5e2-413b-8cde-2a6b4fb78b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:27.155896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde21d7-f5e2-413b-8cde-2a6b4fb78b62 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:27.156255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde21d7-f5e2-413b-8cde-2a6b4fb78b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:27.156534 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde21d7-f5e2-413b-8cde-2a6b4fb78b62 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:27.157333 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde21d7-f5e2-413b-8cde-2a6b4fb78b62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:27.157646 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde21d7-f5e2-413b-8cde-2a6b4fb78b62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:27.159210 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde21d7-f5e2-413b-8cde-2a6b4fb78b62 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:27.192998 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1b8fb6-b32d-4a91-be98-2c6bcaa0b8ce None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:27.197794 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 317/621] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:27.203360 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde21d7-f5e2-413b-8cde-2a6b4fb78b62 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:27.222689 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:27.231036 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ab1c5-08ea-4cfa-a1d7-cf0f172adac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:27.231883 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ab1c5-08ea-4cfa-a1d7-cf0f172adac9 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:27.232422 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ab1c5-08ea-4cfa-a1d7-cf0f172adac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:27.232705 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22ab1c5-08ea-4cfa-a1d7-cf0f172adac9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:27.244382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ab1c5-08ea-4cfa-a1d7-cf0f172adac9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:27.245851 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22ab1c5-08ea-4cfa-a1d7-cf0f172adac9 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:27.265028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-b22ab1c5-08ea-4cfa-a1d7-cf0f172adac9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:27.265028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:27.265028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:27.265028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:27.265028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:27.265028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:27.265028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:27.265028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:27.265028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:27.265028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:27.265028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:27.265028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:27.265028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:27.265028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:27.265028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:27.265982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:27.265982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:27.265982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:27.265982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:27.265982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:27.265982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:27.265982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:27.265982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:27.265982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:27.265982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:27.265982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:27.265982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:27.265982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:27.265982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:27.265982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:27.265982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:27.265982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:27.266868 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:27.266868 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:27.266868 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:27.266868 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:27.266868 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 305/622] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:14:27.276646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ab1c5-08ea-4cfa-a1d7-cf0f172adac9 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:27.365529 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde21d7-f5e2-413b-8cde-2a6b4fb78b62 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:27.389498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22ab1c5-08ea-4cfa-a1d7-cf0f172adac9 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:27.417668 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360529d6-22dd-4039-a273-afc91e43eb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:27.418468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360529d6-22dd-4039-a273-afc91e43eb16 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:27.418750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360529d6-22dd-4039-a273-afc91e43eb16 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:27.419025 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360529d6-22dd-4039-a273-afc91e43eb16 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:27.426702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360529d6-22dd-4039-a273-afc91e43eb16 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:27.428206 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360529d6-22dd-4039-a273-afc91e43eb16 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:27.434172 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472135ed-134c-4655-a1a0-0ccb20a39058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:27.434841 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472135ed-134c-4655-a1a0-0ccb20a39058 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:27.435122 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472135ed-134c-4655-a1a0-0ccb20a39058 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:27.435425 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472135ed-134c-4655-a1a0-0ccb20a39058 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:27.436308 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472135ed-134c-4655-a1a0-0ccb20a39058 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:27.436750 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472135ed-134c-4655-a1a0-0ccb20a39058 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:27.438123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472135ed-134c-4655-a1a0-0ccb20a39058 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:27.444638 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-360529d6-22dd-4039-a273-afc91e43eb16 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 10:14:27.444638 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:27.444638 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:27.444638 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:27.444638 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:27.444638 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:27.444638 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:27.444638 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:27.444638 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:27.444638 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:27.444638 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:27.444638 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:27.444638 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:14:27.444638 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:14:27.444638 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:14:27.444638 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:14:27.445564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:27.445564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:27.445564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:14:27.445564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:27.445564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:14:27.445564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:27.445564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:27.445564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:27.445564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:27.445564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:27.445564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:27.445564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:27.445564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:27.445564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:27.445564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:27.445564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:27.445564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:27.445564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:27.446493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:27.446493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:27.446493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:14:27.446493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:14:27.446493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:14:27.446493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:14:27.446493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:14:27.446493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:14:27.446493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 10:14:27.446493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:27.446493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 318/623] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jun 30 10:14:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:27.447585 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505ae89-90b7-4be0-bd35-1443f33a8be9 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:27.452198 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505ae89-90b7-4be0-bd35-1443f33a8be9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:27.452198 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c505ae89-90b7-4be0-bd35-1443f33a8be9 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:27.452198 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 319/624] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:27.452473 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 306/625] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:14:27.456458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505ae89-90b7-4be0-bd35-1443f33a8be9 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:27.457563 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324121b-6bd7-4cfb-a5a6-e4852fea893e None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:27.457857 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324121b-6bd7-4cfb-a5a6-e4852fea893e None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:27.458135 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e324121b-6bd7-4cfb-a5a6-e4852fea893e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:27.488250 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-e324121b-6bd7-4cfb-a5a6-e4852fea893e None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:27.490256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-e324121b-6bd7-4cfb-a5a6-e4852fea893e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:27.522167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:27.526117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:27.567421 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 307/626] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:27.568645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324121b-6bd7-4cfb-a5a6-e4852fea893e None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:27.595501 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c505ae89-90b7-4be0-bd35-1443f33a8be9 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:27.644457 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d7d23-df40-4eed-860e-a9cf5f3da3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:27.645056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d7d23-df40-4eed-860e-a9cf5f3da3cf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:27.645337 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d7d23-df40-4eed-860e-a9cf5f3da3cf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:27.645610 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d7d23-df40-4eed-860e-a9cf5f3da3cf None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:27.646463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d7d23-df40-4eed-860e-a9cf5f3da3cf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:27.647292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d7d23-df40-4eed-860e-a9cf5f3da3cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:27.649350 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d7d23-df40-4eed-860e-a9cf5f3da3cf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:27.677858 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 320/627] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jun 30 10:14:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:27.677858 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d7d23-df40-4eed-860e-a9cf5f3da3cf None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:27.689833 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e324121b-6bd7-4cfb-a5a6-e4852fea893e None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:27.727597 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f516c5b-e536-4c21-92e4-7a8d17375fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:27.728820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f516c5b-e536-4c21-92e4-7a8d17375fcb None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:27.729336 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f516c5b-e536-4c21-92e4-7a8d17375fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:27.729806 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f516c5b-e536-4c21-92e4-7a8d17375fcb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:27.738136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f516c5b-e536-4c21-92e4-7a8d17375fcb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:27.739945 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f516c5b-e536-4c21-92e4-7a8d17375fcb None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:27.755101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-8f516c5b-e536-4c21-92e4-7a8d17375fcb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:27.755101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:27.755101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:27.755101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:27.755101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:27.755101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:27.755101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:27.755101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:27.755101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:27.755101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:27.755101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:27.755101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:27.755101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:14:27.755101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:14:27.755101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:14:27.756066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:14:27.756066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:27.756066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:27.756066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:27.756066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:27.756066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:27.756066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:27.756066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:27.756066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:27.756066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:27.756066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:27.756066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:27.756066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:27.756066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:27.756066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:27.756066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:27.756066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:27.756983 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:14:27.756983 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:14:27.756983 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:14:27.756983 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:14:27.756983 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:14:27.756983 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:14:27.756983 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:27.756983 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:27.759575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 308/628] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:27.760658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f516c5b-e536-4c21-92e4-7a8d17375fcb None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:27.793702 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:27.801317 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d7d23-df40-4eed-860e-a9cf5f3da3cf None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:27.802884 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:27.837005 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75441b4c-549c-4dc9-9386-e18a580b10c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:27.838072 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75441b4c-549c-4dc9-9386-e18a580b10c3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:27.838353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75441b4c-549c-4dc9-9386-e18a580b10c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:27.838623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75441b4c-549c-4dc9-9386-e18a580b10c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:27.849415 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75441b4c-549c-4dc9-9386-e18a580b10c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:27.857491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-75441b4c-549c-4dc9-9386-e18a580b10c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:27.857491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:27.857491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:27.857491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:27.857491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:27.857491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:27.857491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:27.857491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:27.857491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:27.857491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:27.857491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:27.857491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:27.857491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:27.857491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:27.857491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:27.857491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:27.858449 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:27.858449 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:27.858449 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:27.858449 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:27.858449 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:27.858449 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:27.858449 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:27.858449 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:27.858449 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:27.858449 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:27.858449 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:27.858449 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:27.858449 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:27.858449 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:27.858449 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:27.858449 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:27.859327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:27.859327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:27.859327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:27.859327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:27.859327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:27.859327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:27.859327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:27.859327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:27.859327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:27.859327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:27.859327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:27.859327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:27.867554 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 321/629] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jun 30 10:14:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:27.867554 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75441b4c-549c-4dc9-9386-e18a580b10c3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:27.883254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f516c5b-e536-4c21-92e4-7a8d17375fcb None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:27.922322 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c494ed-49d9-4626-b36d-0b0cc4678802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:27.922991 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c494ed-49d9-4626-b36d-0b0cc4678802 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:27.923255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c494ed-49d9-4626-b36d-0b0cc4678802 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:27.923562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c494ed-49d9-4626-b36d-0b0cc4678802 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:27.974052 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75441b4c-549c-4dc9-9386-e18a580b10c3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:27.996821 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c494ed-49d9-4626-b36d-0b0cc4678802 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:27.998297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c494ed-49d9-4626-b36d-0b0cc4678802 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:28.013091 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16904709-5d10-43e2-b8d3-7f192a3634ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:28.014018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16904709-5d10-43e2-b8d3-7f192a3634ab None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:28.014439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16904709-5d10-43e2-b8d3-7f192a3634ab None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:28.014877 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16904709-5d10-43e2-b8d3-7f192a3634ab None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:28.015956 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16904709-5d10-43e2-b8d3-7f192a3634ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:28.016426 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16904709-5d10-43e2-b8d3-7f192a3634ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:28.019303 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16904709-5d10-43e2-b8d3-7f192a3634ab None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:28.025880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:28.040529 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 322/630] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:14:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:28.040529 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e72f7-199b-4234-9853-e0b9c167fa5f None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:28.040529 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e72f7-199b-4234-9853-e0b9c167fa5f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:28.040529 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47e72f7-199b-4234-9853-e0b9c167fa5f None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:28.040529 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 323/631] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:28.040529 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e72f7-199b-4234-9853-e0b9c167fa5f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:28.067428 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:28.110260 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 309/632] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:14:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:14:28.115373 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c494ed-49d9-4626-b36d-0b0cc4678802 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:28.153740 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47e72f7-199b-4234-9853-e0b9c167fa5f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:28.188800 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd6efc-1e7c-404f-867a-b9149fb8c6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:28.189396 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd6efc-1e7c-404f-867a-b9149fb8c6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:28.189685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd6efc-1e7c-404f-867a-b9149fb8c6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:28.189966 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd6efc-1e7c-404f-867a-b9149fb8c6e9 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:28.190712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd6efc-1e7c-404f-867a-b9149fb8c6e9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:28.191021 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd6efc-1e7c-404f-867a-b9149fb8c6e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:28.192430 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd6efc-1e7c-404f-867a-b9149fb8c6e9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:28.200530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 324/633] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jun 30 10:14:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:28.201812 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55572a-9bb8-4e86-9781-4a58f1799b48 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:28.202101 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55572a-9bb8-4e86-9781-4a58f1799b48 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:28.202378 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe55572a-9bb8-4e86-9781-4a58f1799b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:28.219774 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-fe55572a-9bb8-4e86-9781-4a58f1799b48 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:28.221781 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fe55572a-9bb8-4e86-9781-4a58f1799b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:28.229028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c494ed-49d9-4626-b36d-0b0cc4678802 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:28.257230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:28.268119 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a46872-14bf-4ba5-af32-416d6e91171a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:28.270630 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a46872-14bf-4ba5-af32-416d6e91171a None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:28.270630 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a46872-14bf-4ba5-af32-416d6e91171a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:28.270630 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a46872-14bf-4ba5-af32-416d6e91171a None admin] PATH_INFO: `/v3/projects/50902e0539434c82b505dc5009aeaf85/users/07de926cb8594b098f79111dab8a9cc6/roles/99d8cd9cd87e472290bf0de2710b5857` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:28.287650 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a46872-14bf-4ba5-af32-416d6e91171a None admin] RBAC: Authorizing `identity:create_grant(project_id=50902e0539434c82b505dc5009aeaf85, user_id=07de926cb8594b098f79111dab8a9cc6, role_id=99d8cd9cd87e472290bf0de2710b5857)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:28.289117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a46872-14bf-4ba5-af32-416d6e91171a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:28.293621 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 325/634] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:28.294640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55572a-9bb8-4e86-9781-4a58f1799b48 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:28.311143 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:28.350050 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 310/635] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jun 30 10:14:28 2022] PUT /identity/v3/projects/50902e0539434c82b505dc5009aeaf85/users/07de926cb8594b098f79111dab8a9cc6/roles/99d8cd9cd87e472290bf0de2710b5857 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:14:28.350763 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a46872-14bf-4ba5-af32-416d6e91171a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:28.421593 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe55572a-9bb8-4e86-9781-4a58f1799b48 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:28.462327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36948ec-f777-4b00-931f-e486aa70f47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:28.463051 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36948ec-f777-4b00-931f-e486aa70f47f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:28.463366 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36948ec-f777-4b00-931f-e486aa70f47f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:28.463889 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36948ec-f777-4b00-931f-e486aa70f47f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:28.466792 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a46872-14bf-4ba5-af32-416d6e91171a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:28.475285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36948ec-f777-4b00-931f-e486aa70f47f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:28.476763 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36948ec-f777-4b00-931f-e486aa70f47f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:28.503008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-e36948ec-f777-4b00-931f-e486aa70f47f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:28.503008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:28.503008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:28.503008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:28.503008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:28.503008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:28.503008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:28.503008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:28.503008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:28.503008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:28.503008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:28.503008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:28.503008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:28.503008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:28.503008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:28.503990 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:28.503990 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:28.503990 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:28.503990 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:28.503990 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:28.503990 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:28.503990 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:28.503990 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:28.503990 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:28.503990 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:28.503990 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:28.503990 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:28.503990 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:28.503990 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:28.503990 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:28.503990 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:28.503990 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:28.504893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:28.504893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:28.504893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:28.504893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:28.508294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 326/636] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:28.509597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36948ec-f777-4b00-931f-e486aa70f47f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:28.516335 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9188d0-684b-4467-bd1e-d32e04b07156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:28.517151 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9188d0-684b-4467-bd1e-d32e04b07156 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:28.518085 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9188d0-684b-4467-bd1e-d32e04b07156 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:28.518557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9188d0-684b-4467-bd1e-d32e04b07156 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:28.529846 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9188d0-684b-4467-bd1e-d32e04b07156 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:28.531445 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9188d0-684b-4467-bd1e-d32e04b07156 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:28.549836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-de9188d0-684b-4467-bd1e-d32e04b07156 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:28.549836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:28.549836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:28.549836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:28.549836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:28.549836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:28.549836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:28.549836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:28.549836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:28.549836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:28.549836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:28.549836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:28.549836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:28.549836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:28.549836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:28.550805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:28.550805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:28.550805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:28.550805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:28.550805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:28.550805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:28.550805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:28.550805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:28.550805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:28.550805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:28.550805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:28.550805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:28.550805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:28.550805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:28.550805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:28.550805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:28.550805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:28.551747 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:28.551747 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:28.551747 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:28.551747 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:28.551747 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 311/637] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:28.552467 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c3008-0ec9-43e6-8254-12765872cec3 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:28.552630 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c3008-0ec9-43e6-8254-12765872cec3 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:28.553065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281c3008-0ec9-43e6-8254-12765872cec3 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:28.558061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 312/638] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:28.558061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c3008-0ec9-43e6-8254-12765872cec3 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:28.589212 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:28.594017 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:28.630324 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36948ec-f777-4b00-931f-e486aa70f47f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:28.670768 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281c3008-0ec9-43e6-8254-12765872cec3 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:28.671786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fb7d5-7f88-4ba1-a11c-824de971de45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:28.672835 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fb7d5-7f88-4ba1-a11c-824de971de45 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:28.673233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fb7d5-7f88-4ba1-a11c-824de971de45 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:28.673648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fb7d5-7f88-4ba1-a11c-824de971de45 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:28.689354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879fb7d5-7f88-4ba1-a11c-824de971de45 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:28.697411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-879fb7d5-7f88-4ba1-a11c-824de971de45 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:28.697411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:28.697411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:28.697411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:28.697411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:28.697411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:28.697411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:28.697411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:28.697411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:28.697411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:28.697411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:28.697411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:28.697411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:28.697411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:28.697411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:28.697411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:28.698453 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:28.698453 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:28.698453 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:28.698453 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:28.698453 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:28.698453 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:28.698453 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:28.698453 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:28.698453 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:28.698453 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:28.698453 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:28.698453 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:28.698453 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:28.698453 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:28.698453 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:28.698453 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:28.699302 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:28.699302 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:28.699302 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:28.699302 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:28.699302 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:28.699302 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:28.699302 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:28.699302 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:28.699302 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:28.699302 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:28.699302 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:28.699302 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:28.699302 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 327/639] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jun 30 10:14:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:28.700014 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fb7d5-7f88-4ba1-a11c-824de971de45 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:28.719795 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee925c76-0b02-4223-8f7f-521bba80a32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:28.720375 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee925c76-0b02-4223-8f7f-521bba80a32c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:28.721265 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee925c76-0b02-4223-8f7f-521bba80a32c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:28.721551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee925c76-0b02-4223-8f7f-521bba80a32c None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:28.722292 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee925c76-0b02-4223-8f7f-521bba80a32c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:28.722601 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee925c76-0b02-4223-8f7f-521bba80a32c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:28.724031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee925c76-0b02-4223-8f7f-521bba80a32c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:28.736313 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 313/640] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:28.741775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f09a33-2915-4934-9584-750803950d6f None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:28.742075 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f09a33-2915-4934-9584-750803950d6f None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:28.742363 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f09a33-2915-4934-9584-750803950d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:28.758697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-25f09a33-2915-4934-9584-750803950d6f None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:28.759934 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-25f09a33-2915-4934-9584-750803950d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:28.809592 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fb7d5-7f88-4ba1-a11c-824de971de45 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:28.832823 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 314/641] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:28.836146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f09a33-2915-4934-9584-750803950d6f None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:28.841420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:28.851976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69711a16-8f9a-4d98-a342-6a187090876f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:28.852651 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69711a16-8f9a-4d98-a342-6a187090876f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:28.853167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69711a16-8f9a-4d98-a342-6a187090876f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:28.853311 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69711a16-8f9a-4d98-a342-6a187090876f None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:28.854134 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69711a16-8f9a-4d98-a342-6a187090876f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:28.854453 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69711a16-8f9a-4d98-a342-6a187090876f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:28.856306 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69711a16-8f9a-4d98-a342-6a187090876f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:28.860679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:28.869982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 328/642] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:28.871380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69711a16-8f9a-4d98-a342-6a187090876f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:28.951813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f09a33-2915-4934-9584-750803950d6f None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:28.980710 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69711a16-8f9a-4d98-a342-6a187090876f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:28.991494 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79e7d6-63a4-407e-93f7-177d4f684ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:28.992366 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79e7d6-63a4-407e-93f7-177d4f684ddc None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:28.992804 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79e7d6-63a4-407e-93f7-177d4f684ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:28.993220 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d79e7d6-63a4-407e-93f7-177d4f684ddc None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:28.994103 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d79e7d6-63a4-407e-93f7-177d4f684ddc None admin] RBAC: Adding query filter params (name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:28.994563 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d79e7d6-63a4-407e-93f7-177d4f684ddc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:28.996102 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d79e7d6-63a4-407e-93f7-177d4f684ddc None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:29.009991 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 315/643] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jun 30 10:14:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:29.011061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79e7d6-63a4-407e-93f7-177d4f684ddc None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:29.020403 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5b784-85a3-4048-9801-f74b601aea98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:29.021866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5b784-85a3-4048-9801-f74b601aea98 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:29.022149 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5b784-85a3-4048-9801-f74b601aea98 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:29.022425 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc5b784-85a3-4048-9801-f74b601aea98 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:29.029347 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5b784-85a3-4048-9801-f74b601aea98 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:29.030784 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc5b784-85a3-4048-9801-f74b601aea98 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:29.047983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-acc5b784-85a3-4048-9801-f74b601aea98 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:29.047983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:29.047983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:29.047983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:29.047983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:29.047983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:29.047983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:29.047983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:29.047983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:29.047983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:29.047983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:29.047983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:29.047983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:14:29.047983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:14:29.047983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:14:29.049552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:14:29.049552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:29.049552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:29.049552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:29.049552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:29.049552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:29.049552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:29.049552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:29.049552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:29.049552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:29.049552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:29.049552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:29.049552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:29.049552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:29.049552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:29.049552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:29.049552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:29.050451 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:14:29.050451 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:14:29.050451 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:14:29.050451 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:14:29.050451 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:14:29.050451 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:14:29.050451 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:29.050451 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:29.050451 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 329/644] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:29.050960 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5b784-85a3-4048-9801-f74b601aea98 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:29.105281 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:29.106486 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:29.125766 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d79e7d6-63a4-407e-93f7-177d4f684ddc None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:29.163795 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc5b784-85a3-4048-9801-f74b601aea98 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:29.164830 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7dbab7-3500-4cf8-837e-5b4df63d29c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:29.165785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7dbab7-3500-4cf8-837e-5b4df63d29c2 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:29.166824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7dbab7-3500-4cf8-837e-5b4df63d29c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:29.167104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7dbab7-3500-4cf8-837e-5b4df63d29c2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:29.180883 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7dbab7-3500-4cf8-837e-5b4df63d29c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:29.188637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-ed7dbab7-3500-4cf8-837e-5b4df63d29c2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:29.188637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:29.188637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:29.188637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:29.188637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:29.188637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:29.188637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:29.188637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:29.188637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:29.188637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:29.188637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:29.188637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:29.188637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:29.188637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:29.188637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:29.188637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:29.189968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:29.189968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:29.189968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:29.189968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:29.189968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:29.189968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:29.189968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:29.189968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:29.189968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:29.189968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:29.189968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:29.189968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:29.189968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:29.189968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:29.189968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:29.189968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:29.190859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:29.190859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:29.190859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:29.190859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:29.190859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:29.190859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:29.190859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:29.190859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:29.190859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:29.190859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:29.190859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:29.190859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:29.190859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 316/645] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jun 30 10:14:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:29.190859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7dbab7-3500-4cf8-837e-5b4df63d29c2 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:29.203444 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1a67a-3227-44a0-9b4b-a111baa478f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:29.204243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1a67a-3227-44a0-9b4b-a111baa478f3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:29.204528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1a67a-3227-44a0-9b4b-a111baa478f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:29.204833 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1a67a-3227-44a0-9b4b-a111baa478f3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:29.212945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba1a67a-3227-44a0-9b4b-a111baa478f3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:29.214401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba1a67a-3227-44a0-9b4b-a111baa478f3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:29.229407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-9ba1a67a-3227-44a0-9b4b-a111baa478f3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 30 10:14:29.229407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:29.229407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:29.229407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:29.229407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:29.229407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:29.229407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:29.229407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:29.229407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:29.229407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:29.229407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:29.229407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:29.229407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:14:29.229407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:14:29.229407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:14:29.229407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:14:29.230349 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:29.230349 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:29.230349 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:14:29.230349 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:29.230349 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:14:29.230349 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:29.230349 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:29.230349 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:29.230349 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:29.230349 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:29.230349 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:29.230349 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:29.230349 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:29.230349 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:29.230349 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:29.230349 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:29.230349 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:29.230349 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:29.231332 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:29.231332 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:29.231332 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:14:29.231332 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:14:29.231332 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:14:29.231332 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:14:29.231332 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:14:29.231332 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:14:29.231332 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 30 10:14:29.231332 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:29.231332 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 330/646] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:14:29 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:29.232528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1a67a-3227-44a0-9b4b-a111baa478f3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:29.313825 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7dbab7-3500-4cf8-837e-5b4df63d29c2 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:29.341981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1a67a-3227-44a0-9b4b-a111baa478f3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:29.349014 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:29.350983 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c6fd3-ce29-430f-9285-db60d789fa1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:29.352170 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c6fd3-ce29-430f-9285-db60d789fa1c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:29.352458 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c6fd3-ce29-430f-9285-db60d789fa1c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:29.352945 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9c6fd3-ce29-430f-9285-db60d789fa1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:29.362048 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:29.364334 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c6fd3-ce29-430f-9285-db60d789fa1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:29.365873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9c6fd3-ce29-430f-9285-db60d789fa1c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:29.375382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4599c16-16e8-438f-8522-77db933c32c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:29.376051 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4599c16-16e8-438f-8522-77db933c32c7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:29.376422 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4599c16-16e8-438f-8522-77db933c32c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:29.376633 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4599c16-16e8-438f-8522-77db933c32c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:29.378068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4599c16-16e8-438f-8522-77db933c32c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:29.378774 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4599c16-16e8-438f-8522-77db933c32c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:29.380694 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4599c16-16e8-438f-8522-77db933c32c7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:29.383337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-6b9c6fd3-ce29-430f-9285-db60d789fa1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:29.383337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:29.383337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:29.383337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:29.383337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:29.383337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:29.383337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:29.383337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:29.383337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:29.383337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:29.383337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:29.383337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:29.383337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:29.383337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:29.383337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:29.390382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:29.390382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:29.390382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:29.390382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:29.390382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:29.390382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:29.390382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:29.390382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:29.390382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:29.390382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:29.390382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:29.390382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:29.390382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:29.390382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:29.390382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:29.390382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:29.390382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:29.391357 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:29.391357 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:29.391357 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:29.391357 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:29.391357 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 317/647] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:29.391357 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c6fd3-ce29-430f-9285-db60d789fa1c None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:29.398317 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 331/648] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:14:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:29.399291 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4599c16-16e8-438f-8522-77db933c32c7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:29.509917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9c6fd3-ce29-430f-9285-db60d789fa1c None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:29.513039 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4599c16-16e8-438f-8522-77db933c32c7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:29.546054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470652d-e9d4-491a-b5ac-a00c1b8d4b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:29.546724 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470652d-e9d4-491a-b5ac-a00c1b8d4b57 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:29.547005 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470652d-e9d4-491a-b5ac-a00c1b8d4b57 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:29.547289 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470652d-e9d4-491a-b5ac-a00c1b8d4b57 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:29.547611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371bd9aa-70ce-4c4c-81b9-b99390b499d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:29.548046 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470652d-e9d4-491a-b5ac-a00c1b8d4b57 None admin] RBAC: Adding query filter params (name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:29.548306 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371bd9aa-70ce-4c4c-81b9-b99390b499d4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:29.548424 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470652d-e9d4-491a-b5ac-a00c1b8d4b57 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:29.549096 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371bd9aa-70ce-4c4c-81b9-b99390b499d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:29.549365 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371bd9aa-70ce-4c4c-81b9-b99390b499d4 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:29.550197 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371bd9aa-70ce-4c4c-81b9-b99390b499d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:29.550296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7470652d-e9d4-491a-b5ac-a00c1b8d4b57 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:29.550509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371bd9aa-70ce-4c4c-81b9-b99390b499d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:29.551970 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371bd9aa-70ce-4c4c-81b9-b99390b499d4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:29.563480 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 318/649] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jun 30 10:14:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:29.564927 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470652d-e9d4-491a-b5ac-a00c1b8d4b57 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:29.574335 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 332/650] 10.222.0.47 () {56 vars in 1137 bytes} [Thu Jun 30 10:14:29 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:29.574335 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371bd9aa-70ce-4c4c-81b9-b99390b499d4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:29.599633 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:29.623759 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:29.672108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470652d-e9d4-491a-b5ac-a00c1b8d4b57 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:29.677885 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371bd9aa-70ce-4c4c-81b9-b99390b499d4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:29.707089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060b0e6-85a3-4306-8f77-839b82e5772c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:29.707813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060b0e6-85a3-4306-8f77-839b82e5772c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:29.708103 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060b0e6-85a3-4306-8f77-839b82e5772c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:29.708385 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060b0e6-85a3-4306-8f77-839b82e5772c None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:29.709109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060b0e6-85a3-4306-8f77-839b82e5772c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:29.709507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060b0e6-85a3-4306-8f77-839b82e5772c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:29.710840 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060b0e6-85a3-4306-8f77-839b82e5772c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:29.714157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3288e7-8e69-4068-865a-5454142729ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:29.714803 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3288e7-8e69-4068-865a-5454142729ad None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:29.715085 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3288e7-8e69-4068-865a-5454142729ad None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:29.715392 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3288e7-8e69-4068-865a-5454142729ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:29.716217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3288e7-8e69-4068-865a-5454142729ad None admin] RBAC: Adding query filter params (role.id=99d8cd9cd87e472290bf0de2710b5857, scope.project.id=50902e0539434c82b505dc5009aeaf85, user.id=71f6d7715e6f49ffb9fdf6e1307bbd51) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:29.716523 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3288e7-8e69-4068-865a-5454142729ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:29.718486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3288e7-8e69-4068-865a-5454142729ad None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:29.727724 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 319/651] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:29.727724 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060b0e6-85a3-4306-8f77-839b82e5772c None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:29.730418 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 333/652] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:14:29 2022] GET /identity/v3/role_assignments?user.id=71f6d7715e6f49ffb9fdf6e1307bbd51&scope.project.id=50902e0539434c82b505dc5009aeaf85&role.id=99d8cd9cd87e472290bf0de2710b5857 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:29.734149 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3288e7-8e69-4068-865a-5454142729ad None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:29.840266 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060b0e6-85a3-4306-8f77-839b82e5772c None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:29.856073 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:29.865450 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:29.875963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3288e7-8e69-4068-865a-5454142729ad None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:29.889798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273eb490-ea0c-4ccd-a932-9a9850aa73b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:29.891005 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273eb490-ea0c-4ccd-a932-9a9850aa73b8 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:29.891679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273eb490-ea0c-4ccd-a932-9a9850aa73b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:29.898066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273eb490-ea0c-4ccd-a932-9a9850aa73b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:29.904437 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273eb490-ea0c-4ccd-a932-9a9850aa73b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:29.906206 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273eb490-ea0c-4ccd-a932-9a9850aa73b8 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:29.924566 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d6933-34d5-4e7b-9e54-076c4429c4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:29.925285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d6933-34d5-4e7b-9e54-076c4429c4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:29.925574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d6933-34d5-4e7b-9e54-076c4429c4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:29.925855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58d6933-34d5-4e7b-9e54-076c4429c4d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:29.929790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-273eb490-ea0c-4ccd-a932-9a9850aa73b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:29.929790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:29.929790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:29.929790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:29.929790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:29.929790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:29.929790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:29.929790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:29.929790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:29.929790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:29.929790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:29.929790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:29.929790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:29.929790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:29.929790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:29.930653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:29.930653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:29.930653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:29.930653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:29.930653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:29.930653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:29.930653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:29.930653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:29.930653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:29.930653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:29.930653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:29.930653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:29.930653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:29.930653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:29.930653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:29.930653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:29.930653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:29.931591 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:29.931591 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:29.931591 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:29.931591 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:29.932283 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 320/653] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:29.933147 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273eb490-ea0c-4ccd-a932-9a9850aa73b8 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:30.015006 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d6933-34d5-4e7b-9e54-076c4429c4d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:30.016672 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58d6933-34d5-4e7b-9e54-076c4429c4d5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:30.067141 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273eb490-ea0c-4ccd-a932-9a9850aa73b8 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:30.105667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09d9fc-ad0b-4f0b-885b-fb4e53e570df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:30.106497 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09d9fc-ad0b-4f0b-885b-fb4e53e570df None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:30.106786 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09d9fc-ad0b-4f0b-885b-fb4e53e570df None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:30.107163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09d9fc-ad0b-4f0b-885b-fb4e53e570df None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:30.116112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09d9fc-ad0b-4f0b-885b-fb4e53e570df None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:30.117770 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09d9fc-ad0b-4f0b-885b-fb4e53e570df None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:30.128977 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:30.135291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-6d09d9fc-ad0b-4f0b-885b-fb4e53e570df None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 10:14:30.135291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:30.135291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:30.135291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:30.135291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:30.135291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:30.135291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:30.135291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:30.135291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:30.135291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:30.135291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:30.135291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:30.135291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:14:30.135291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:14:30.135291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:14:30.135291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:14:30.138658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:30.138658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:30.138658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:14:30.138658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:30.138658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:14:30.138658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:30.138658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:30.138658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:30.138658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:30.138658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:30.138658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:30.138658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:30.138658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:30.138658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:30.138658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:30.138658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:30.138658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:30.138658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:30.139642 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:30.139642 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:30.139642 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:14:30.139642 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:14:30.139642 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:14:30.139642 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:14:30.139642 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:14:30.139642 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:14:30.139642 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 10:14:30.139642 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:30.139642 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 321/654] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:14:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:30.139642 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09d9fc-ad0b-4f0b-885b-fb4e53e570df None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:30.149142 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 334/655] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:14:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:14:30.150032 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d6933-34d5-4e7b-9e54-076c4429c4d5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:30.179799 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:30.261798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09d9fc-ad0b-4f0b-885b-fb4e53e570df None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:30.291170 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58d6933-34d5-4e7b-9e54-076c4429c4d5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:30.306095 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee18e2-2262-43aa-9042-79e1252f3938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:30.307048 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee18e2-2262-43aa-9042-79e1252f3938 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:30.307646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee18e2-2262-43aa-9042-79e1252f3938 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:30.308071 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee18e2-2262-43aa-9042-79e1252f3938 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:30.309460 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee18e2-2262-43aa-9042-79e1252f3938 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:30.310022 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee18e2-2262-43aa-9042-79e1252f3938 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:30.311743 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee18e2-2262-43aa-9042-79e1252f3938 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:30.328059 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 322/656] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:30.329220 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee18e2-2262-43aa-9042-79e1252f3938 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:30.330471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a3816-87fc-47ff-bf03-2db0f9d58173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:30.331326 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a3816-87fc-47ff-bf03-2db0f9d58173 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:30.331717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a3816-87fc-47ff-bf03-2db0f9d58173 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:30.332129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a3816-87fc-47ff-bf03-2db0f9d58173 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:30.333092 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a3816-87fc-47ff-bf03-2db0f9d58173 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:30.333504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a3816-87fc-47ff-bf03-2db0f9d58173 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:30.335060 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a3816-87fc-47ff-bf03-2db0f9d58173 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:30.362889 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 335/657] 10.222.0.47 () {56 vars in 1137 bytes} [Thu Jun 30 10:14:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:30.363906 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a3816-87fc-47ff-bf03-2db0f9d58173 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:30.413689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:30.440176 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:30.456256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee18e2-2262-43aa-9042-79e1252f3938 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:30.472143 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a3816-87fc-47ff-bf03-2db0f9d58173 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:30.493475 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5fa23-0638-47b8-b494-bea58262f86c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:30.494279 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5fa23-0638-47b8-b494-bea58262f86c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:30.494567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5fa23-0638-47b8-b494-bea58262f86c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:30.494851 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5fa23-0638-47b8-b494-bea58262f86c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:30.507320 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed5fa23-0638-47b8-b494-bea58262f86c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:30.508877 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed5fa23-0638-47b8-b494-bea58262f86c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:30.514048 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a765a-649d-4d00-8034-883259847ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:30.514869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a765a-649d-4d00-8034-883259847ab6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:30.515152 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a765a-649d-4d00-8034-883259847ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:30.515799 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a765a-649d-4d00-8034-883259847ab6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:30.525613 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a765a-649d-4d00-8034-883259847ab6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:30.525854 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a765a-649d-4d00-8034-883259847ab6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:30.528286 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-0ed5fa23-0638-47b8-b494-bea58262f86c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:30.528286 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:30.528286 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:30.528286 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:30.528286 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:30.528286 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:30.528286 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:30.528286 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:30.528286 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:30.528286 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:30.528286 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:30.528286 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:30.528286 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:30.528286 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:30.528286 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:30.529168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:30.529168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:30.529168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:30.529168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:30.529168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:30.529168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:30.529168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:30.529168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:30.529168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:30.529168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:30.529168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:30.529168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:30.529168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:30.529168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:30.529168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:30.529168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:30.529168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:30.535493 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:30.535493 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:30.535493 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:30.535493 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:30.535493 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 323/658] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:30.535493 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5fa23-0638-47b8-b494-bea58262f86c None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:30.542211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-6b5a765a-649d-4d00-8034-883259847ab6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 10:14:30.542211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:30.542211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:30.542211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:30.542211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:30.542211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:30.542211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:30.542211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:30.542211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:30.542211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:30.542211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:30.542211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:30.542211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:14:30.542211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:14:30.542211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:14:30.542211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:14:30.543116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:30.543116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:30.543116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:14:30.543116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:30.543116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:14:30.543116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:30.543116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:30.543116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:30.543116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:30.543116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:30.543116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:30.543116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:30.543116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:30.543116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:30.543116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:30.543116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:30.543116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:30.543116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:30.544171 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:30.544171 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:30.544171 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:14:30.544171 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:14:30.544171 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:14:30.544171 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:14:30.544171 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:14:30.544171 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:14:30.544171 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 30 10:14:30.544171 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:30.547558 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 336/659] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jun 30 10:14:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:30.549981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25e366-bbc7-48cb-916d-56ed2e000711 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:30.550500 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25e366-bbc7-48cb-916d-56ed2e000711 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:30.550855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25e366-bbc7-48cb-916d-56ed2e000711 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:30.552275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 337/660] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:30.553020 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25e366-bbc7-48cb-916d-56ed2e000711 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:30.647525 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5fa23-0638-47b8-b494-bea58262f86c None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:30.657597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25e366-bbc7-48cb-916d-56ed2e000711 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:30.665404 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:30.684437 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eee757-7b0d-4d54-92e9-d11442bb5db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:30.685374 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eee757-7b0d-4d54-92e9-d11442bb5db9 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:30.685862 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eee757-7b0d-4d54-92e9-d11442bb5db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:30.686360 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eee757-7b0d-4d54-92e9-d11442bb5db9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:30.693689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5716d4-cf24-44cd-8920-30c3069acadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:30.694381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5716d4-cf24-44cd-8920-30c3069acadd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:30.694662 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5716d4-cf24-44cd-8920-30c3069acadd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:30.694941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5716d4-cf24-44cd-8920-30c3069acadd None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:30.695419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:30.696102 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5716d4-cf24-44cd-8920-30c3069acadd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:30.696414 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5716d4-cf24-44cd-8920-30c3069acadd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:30.698117 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5716d4-cf24-44cd-8920-30c3069acadd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:30.710713 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 338/661] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:30.711445 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5716d4-cf24-44cd-8920-30c3069acadd None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:30.761787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eee757-7b0d-4d54-92e9-d11442bb5db9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:30.763399 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eee757-7b0d-4d54-92e9-d11442bb5db9 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:30.815312 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5716d4-cf24-44cd-8920-30c3069acadd None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:30.850646 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88998459-2b64-4f23-b790-d3f0d7a84805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:30.851357 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88998459-2b64-4f23-b790-d3f0d7a84805 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:30.851646 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88998459-2b64-4f23-b790-d3f0d7a84805 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:30.851920 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88998459-2b64-4f23-b790-d3f0d7a84805 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:30.852756 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88998459-2b64-4f23-b790-d3f0d7a84805 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:30.853088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88998459-2b64-4f23-b790-d3f0d7a84805 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:30.854489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88998459-2b64-4f23-b790-d3f0d7a84805 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:30.869735 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 324/662] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:14:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:14:30.871220 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fa8ba-df9b-4d93-b375-dc7d2de335e5 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:30.871682 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fa8ba-df9b-4d93-b375-dc7d2de335e5 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:30.872103 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fa8ba-df9b-4d93-b375-dc7d2de335e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:30.877862 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 339/663] 10.222.0.47 () {56 vars in 1139 bytes} [Thu Jun 30 10:14:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:30.878687 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88998459-2b64-4f23-b790-d3f0d7a84805 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:30.893928 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-8f5fa8ba-df9b-4d93-b375-dc7d2de335e5 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:30.895445 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-8f5fa8ba-df9b-4d93-b375-dc7d2de335e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:30.905503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:30.928135 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:30.968112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 325/664] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:30.969281 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5fa8ba-df9b-4d93-b375-dc7d2de335e5 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:30.995412 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88998459-2b64-4f23-b790-d3f0d7a84805 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:31.035457 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4714d885-7d20-4332-a00f-82defa1670d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:31.036247 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714d885-7d20-4332-a00f-82defa1670d1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:31.036530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714d885-7d20-4332-a00f-82defa1670d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:31.036799 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714d885-7d20-4332-a00f-82defa1670d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:31.043510 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4714d885-7d20-4332-a00f-82defa1670d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:31.045023 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4714d885-7d20-4332-a00f-82defa1670d1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:31.059231 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-4714d885-7d20-4332-a00f-82defa1670d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:31.059231 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:31.059231 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:31.059231 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:31.059231 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:31.059231 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:31.059231 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:31.059231 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:31.059231 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:31.059231 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:31.059231 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:31.059231 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:31.059231 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:14:31.059231 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:14:31.059231 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:14:31.060173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:14:31.060173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:31.060173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:31.060173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:31.060173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:31.060173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:31.060173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:31.060173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:31.060173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:31.060173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:31.060173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:31.060173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:31.060173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:31.060173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:31.060173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:31.060173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:31.060173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:31.061059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:14:31.061059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:14:31.061059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:14:31.061059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:14:31.061059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:14:31.061059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:14:31.061059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:31.061059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:31.061059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 340/665] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:31.061593 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4714d885-7d20-4332-a00f-82defa1670d1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:31.082554 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5fa8ba-df9b-4d93-b375-dc7d2de335e5 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:31.121035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b8cf1-408c-4455-a199-aa6261b7252d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:31.121850 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b8cf1-408c-4455-a199-aa6261b7252d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:31.122133 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b8cf1-408c-4455-a199-aa6261b7252d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:31.122410 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b8cf1-408c-4455-a199-aa6261b7252d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:31.133613 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b8cf1-408c-4455-a199-aa6261b7252d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:31.135093 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b8cf1-408c-4455-a199-aa6261b7252d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:31.152262 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-5b7b8cf1-408c-4455-a199-aa6261b7252d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:31.152262 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:31.152262 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:31.152262 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:31.152262 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:31.152262 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:31.152262 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:31.152262 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:31.152262 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:31.152262 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:31.152262 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:31.152262 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:31.152262 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:31.152262 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:31.152262 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:31.153386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:31.153386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:31.153386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:31.153386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:31.153386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:31.153386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:31.153386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:31.153386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:31.153386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:31.153386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:31.153386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:31.153386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:31.153386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:31.153386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:31.153386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:31.153386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:31.153386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:31.154512 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:31.154512 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:31.154512 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:31.154512 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:31.156746 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:31.158074 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 326/666] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:31.159076 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b8cf1-408c-4455-a199-aa6261b7252d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:31.176205 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4714d885-7d20-4332-a00f-82defa1670d1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:31.182091 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:31.214746 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ad091-7401-43bd-ba80-7473d428484f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:31.215501 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ad091-7401-43bd-ba80-7473d428484f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:31.215784 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ad091-7401-43bd-ba80-7473d428484f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:31.216057 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ad091-7401-43bd-ba80-7473d428484f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:31.265566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b8cf1-408c-4455-a199-aa6261b7252d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:31.285434 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ad091-7401-43bd-ba80-7473d428484f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:31.287018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ad091-7401-43bd-ba80-7473d428484f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:31.300624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca856ad5-7e07-46b3-8ab4-606888f1609a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:31.301424 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca856ad5-7e07-46b3-8ab4-606888f1609a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:31.301709 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca856ad5-7e07-46b3-8ab4-606888f1609a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:31.301979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca856ad5-7e07-46b3-8ab4-606888f1609a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:31.312520 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca856ad5-7e07-46b3-8ab4-606888f1609a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:31.320820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-ca856ad5-7e07-46b3-8ab4-606888f1609a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:31.320820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:31.320820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:31.320820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:31.320820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:31.320820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:31.320820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:31.320820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:31.320820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:31.320820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:31.320820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:31.320820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:31.320820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:31.320820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:31.320820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:31.320820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:31.321734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:31.321734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:31.321734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:31.321734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:31.321734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:31.321734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:31.321734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:31.321734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:31.321734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:31.321734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:31.321734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:31.321734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:31.321734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:31.321734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:31.321734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:31.321734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:31.322579 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:31.322579 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:31.322579 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:31.322579 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:31.322579 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:31.322579 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:31.322579 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:31.322579 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:31.322579 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:31.322579 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:31.322579 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:31.322579 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:31.322579 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 327/667] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jun 30 10:14:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:31.323962 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca856ad5-7e07-46b3-8ab4-606888f1609a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:31.383205 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:31.400205 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 341/668] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:14:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:14:31.402365 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ad091-7401-43bd-ba80-7473d428484f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:31.420015 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:31.440567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca856ad5-7e07-46b3-8ab4-606888f1609a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:31.475797 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29714c4-d9a9-4eb7-865a-568bc5b7ed02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:31.476472 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29714c4-d9a9-4eb7-865a-568bc5b7ed02 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:31.476763 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29714c4-d9a9-4eb7-865a-568bc5b7ed02 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:31.477045 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29714c4-d9a9-4eb7-865a-568bc5b7ed02 None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:31.477838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29714c4-d9a9-4eb7-865a-568bc5b7ed02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:31.478151 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29714c4-d9a9-4eb7-865a-568bc5b7ed02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:31.479612 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29714c4-d9a9-4eb7-865a-568bc5b7ed02 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:31.497465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 328/669] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:14:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:31.497465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29714c4-d9a9-4eb7-865a-568bc5b7ed02 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:31.523175 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ad091-7401-43bd-ba80-7473d428484f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:31.558307 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91722144-a879-4508-be41-dc0a651e9ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:31.559206 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91722144-a879-4508-be41-dc0a651e9ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:31.559658 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91722144-a879-4508-be41-dc0a651e9ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:31.560095 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91722144-a879-4508-be41-dc0a651e9ee6 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:31.560955 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91722144-a879-4508-be41-dc0a651e9ee6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:31.561418 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91722144-a879-4508-be41-dc0a651e9ee6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:31.562989 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91722144-a879-4508-be41-dc0a651e9ee6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:31.581333 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 342/670] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:31.582230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91722144-a879-4508-be41-dc0a651e9ee6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:31.605461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29714c4-d9a9-4eb7-865a-568bc5b7ed02 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:31.632240 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:31.641412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d8124-6bd9-4813-9c51-104dca2f0073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:31.642096 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d8124-6bd9-4813-9c51-104dca2f0073 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:31.642428 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d8124-6bd9-4813-9c51-104dca2f0073 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:31.642707 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94d8124-6bd9-4813-9c51-104dca2f0073 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:31.647982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d8124-6bd9-4813-9c51-104dca2f0073 None admin] RBAC: Adding query filter params (name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:31.647982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d8124-6bd9-4813-9c51-104dca2f0073 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:31.647982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94d8124-6bd9-4813-9c51-104dca2f0073 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:31.654423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 329/671] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jun 30 10:14:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:31.655448 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d8124-6bd9-4813-9c51-104dca2f0073 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:31.659668 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:31.702536 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91722144-a879-4508-be41-dc0a651e9ee6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:31.738508 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af1796-9b5d-4c89-b2d6-28d5e8e5b971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:31.739318 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af1796-9b5d-4c89-b2d6-28d5e8e5b971 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:31.739944 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af1796-9b5d-4c89-b2d6-28d5e8e5b971 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:31.740221 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22af1796-9b5d-4c89-b2d6-28d5e8e5b971 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:31.750760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af1796-9b5d-4c89-b2d6-28d5e8e5b971 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:31.752293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22af1796-9b5d-4c89-b2d6-28d5e8e5b971 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:31.763949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94d8124-6bd9-4813-9c51-104dca2f0073 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:31.770341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-22af1796-9b5d-4c89-b2d6-28d5e8e5b971 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:31.770341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:31.770341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:31.770341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:31.770341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:31.770341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:31.770341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:31.770341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:31.770341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:31.770341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:31.770341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:31.770341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:31.770341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:31.770341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:31.770341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:31.771250 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:31.771250 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:31.771250 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:31.771250 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:31.771250 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:31.771250 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:31.771250 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:31.771250 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:31.771250 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:31.771250 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:31.771250 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:31.771250 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:31.771250 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:31.771250 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:31.771250 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:31.771250 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:31.771250 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:31.772468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:31.772468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:31.772468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:31.772468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:31.775643 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 343/672] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:31.777076 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af1796-9b5d-4c89-b2d6-28d5e8e5b971 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:31.803224 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72dbda-5ffa-455a-bb96-d1ec857b665f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:31.803929 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72dbda-5ffa-455a-bb96-d1ec857b665f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:31.804233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72dbda-5ffa-455a-bb96-d1ec857b665f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:31.804492 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d72dbda-5ffa-455a-bb96-d1ec857b665f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:31.805322 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72dbda-5ffa-455a-bb96-d1ec857b665f None admin] RBAC: Adding query filter params (role.id=99d8cd9cd87e472290bf0de2710b5857, scope.project.id=50902e0539434c82b505dc5009aeaf85, user.id=c7a61d85908d42698b27b9fc00daa832) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:31.805642 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72dbda-5ffa-455a-bb96-d1ec857b665f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:31.807088 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d72dbda-5ffa-455a-bb96-d1ec857b665f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:31.819463 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 330/673] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:14:31 2022] GET /identity/v3/role_assignments?user.id=c7a61d85908d42698b27b9fc00daa832&scope.project.id=50902e0539434c82b505dc5009aeaf85&role.id=99d8cd9cd87e472290bf0de2710b5857 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:31.821524 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72dbda-5ffa-455a-bb96-d1ec857b665f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:31.890829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:31.898108 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22af1796-9b5d-4c89-b2d6-28d5e8e5b971 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:31.931159 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:31.940625 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79ed19-6cce-4211-83f8-1ec487078210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:31.941457 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79ed19-6cce-4211-83f8-1ec487078210 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:31.941735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79ed19-6cce-4211-83f8-1ec487078210 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:31.942006 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79ed19-6cce-4211-83f8-1ec487078210 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:31.950079 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e79ed19-6cce-4211-83f8-1ec487078210 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:31.952038 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e79ed19-6cce-4211-83f8-1ec487078210 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:31.968179 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-6e79ed19-6cce-4211-83f8-1ec487078210 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:31.968179 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:31.968179 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:31.968179 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:31.968179 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:31.968179 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:31.968179 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:31.968179 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:31.968179 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:31.968179 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:31.968179 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:31.968179 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:31.968179 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:14:31.968179 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:14:31.968179 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:14:31.969298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:14:31.969298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:31.969298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:31.969298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:31.969298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:31.969298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:31.969298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:31.969298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:31.969298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:31.969298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:31.969298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:31.969298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:31.969298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:31.969298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:31.969298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:31.969298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:31.969298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:31.970519 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:14:31.970519 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:14:31.970519 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:14:31.970519 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:14:31.970519 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:14:31.970519 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:14:31.970519 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:31.970519 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:31.974599 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 344/674] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:31.975673 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79ed19-6cce-4211-83f8-1ec487078210 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:31.977520 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d72dbda-5ffa-455a-bb96-d1ec857b665f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:32.020981 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b42c1-6c3a-4c75-8854-21a986377f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:32.022329 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b42c1-6c3a-4c75-8854-21a986377f40 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:32.022615 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b42c1-6c3a-4c75-8854-21a986377f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:32.022940 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b42c1-6c3a-4c75-8854-21a986377f40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:32.034666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b42c1-6c3a-4c75-8854-21a986377f40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:32.036172 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b42c1-6c3a-4c75-8854-21a986377f40 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:32.057274 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-350b42c1-6c3a-4c75-8854-21a986377f40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:32.057274 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:32.057274 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:32.057274 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:32.057274 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:32.057274 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:32.057274 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:32.057274 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:32.057274 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:32.057274 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:32.057274 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:32.057274 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:32.057274 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:32.057274 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:32.057274 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:32.058367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:32.058367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:32.058367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:32.058367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:32.058367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:32.058367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:32.058367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:32.058367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:32.058367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:32.058367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:32.058367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:32.058367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:32.058367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:32.058367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:32.058367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:32.058367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:32.058367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:32.059500 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:32.059500 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:32.059500 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:32.059500 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:32.060703 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 331/675] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:32.061664 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c755e-8d3d-4830-8486-27234f19017a None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:32.062041 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c755e-8d3d-4830-8486-27234f19017a None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:32.062785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824c755e-8d3d-4830-8486-27234f19017a None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:32.067681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 332/676] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:32.067681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c755e-8d3d-4830-8486-27234f19017a None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:32.094970 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79ed19-6cce-4211-83f8-1ec487078210 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:32.132963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ee68b-d7f7-4b0f-8f90-b83516224d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:32.134812 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ee68b-d7f7-4b0f-8f90-b83516224d00 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:32.134812 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ee68b-d7f7-4b0f-8f90-b83516224d00 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:32.134812 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45ee68b-d7f7-4b0f-8f90-b83516224d00 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:32.135113 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ee68b-d7f7-4b0f-8f90-b83516224d00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:32.135369 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ee68b-d7f7-4b0f-8f90-b83516224d00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:32.136755 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45ee68b-d7f7-4b0f-8f90-b83516224d00 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:32.153413 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 345/677] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:32.153413 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ee68b-d7f7-4b0f-8f90-b83516224d00 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:32.170056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:32.182708 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824c755e-8d3d-4830-8486-27234f19017a None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:32.200845 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:32.217409 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb54b419-ff8c-4de8-a157-c8879dc94141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:32.218153 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54b419-ff8c-4de8-a157-c8879dc94141 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:32.218551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54b419-ff8c-4de8-a157-c8879dc94141 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:32.218852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb54b419-ff8c-4de8-a157-c8879dc94141 None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:32.219751 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54b419-ff8c-4de8-a157-c8879dc94141 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:32.220062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54b419-ff8c-4de8-a157-c8879dc94141 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:32.222177 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb54b419-ff8c-4de8-a157-c8879dc94141 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:32.244140 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 333/678] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:14:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:32.256576 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c351e-b23a-4d23-ba98-2b661177109b None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:32.256863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c351e-b23a-4d23-ba98-2b661177109b None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:32.257136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15c351e-b23a-4d23-ba98-2b661177109b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:32.271074 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45ee68b-d7f7-4b0f-8f90-b83516224d00 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:32.275110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-b15c351e-b23a-4d23-ba98-2b661177109b None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:32.277198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-b15c351e-b23a-4d23-ba98-2b661177109b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:32.313638 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07a43e-0af8-4789-9160-06c317bbe7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:32.315237 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07a43e-0af8-4789-9160-06c317bbe7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:32.315549 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07a43e-0af8-4789-9160-06c317bbe7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:32.315834 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07a43e-0af8-4789-9160-06c317bbe7c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:32.316581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07a43e-0af8-4789-9160-06c317bbe7c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:32.316879 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07a43e-0af8-4789-9160-06c317bbe7c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:32.318272 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07a43e-0af8-4789-9160-06c317bbe7c5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:32.334360 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 346/679] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:32.334360 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07a43e-0af8-4789-9160-06c317bbe7c5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:32.355615 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 334/680] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:32.356492 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15c351e-b23a-4d23-ba98-2b661177109b None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:32.424846 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:32.452441 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07a43e-0af8-4789-9160-06c317bbe7c5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:32.472910 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15c351e-b23a-4d23-ba98-2b661177109b None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:32.482629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:32.493962 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630762fa-79ef-4235-8d56-3ca28ad5a7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:32.494750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630762fa-79ef-4235-8d56-3ca28ad5a7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:32.495059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630762fa-79ef-4235-8d56-3ca28ad5a7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:32.495334 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630762fa-79ef-4235-8d56-3ca28ad5a7e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:32.507283 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630762fa-79ef-4235-8d56-3ca28ad5a7e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:32.508220 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45671f2d-0bcf-42ed-9de6-a0a834ddade9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:32.508753 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630762fa-79ef-4235-8d56-3ca28ad5a7e2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:32.508949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45671f2d-0bcf-42ed-9de6-a0a834ddade9 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:32.509168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45671f2d-0bcf-42ed-9de6-a0a834ddade9 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:32.509450 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45671f2d-0bcf-42ed-9de6-a0a834ddade9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:32.510284 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45671f2d-0bcf-42ed-9de6-a0a834ddade9 None admin] RBAC: Adding query filter params (role.id=99d8cd9cd87e472290bf0de2710b5857, scope.project.id=50902e0539434c82b505dc5009aeaf85, user.id=07ab12e5d1d94a009c4d414e5c43ac92) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:32.510600 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45671f2d-0bcf-42ed-9de6-a0a834ddade9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:32.512132 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45671f2d-0bcf-42ed-9de6-a0a834ddade9 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:32.526560 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 335/681] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:14:32 2022] GET /identity/v3/role_assignments?user.id=07ab12e5d1d94a009c4d414e5c43ac92&scope.project.id=50902e0539434c82b505dc5009aeaf85&role.id=99d8cd9cd87e472290bf0de2710b5857 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:32.526560 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45671f2d-0bcf-42ed-9de6-a0a834ddade9 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:32.526859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-630762fa-79ef-4235-8d56-3ca28ad5a7e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:32.526859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:32.526859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:32.526859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:32.526859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:32.526859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:32.526859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:32.526859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:32.526859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:32.526859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:32.526859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:32.526859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:32.526859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:14:32.526859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:14:32.526859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:14:32.527899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:14:32.527899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:32.527899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:32.527899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:32.527899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:32.527899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:32.527899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:32.527899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:32.527899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:32.527899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:32.527899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:32.527899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:32.527899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:32.527899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:32.527899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:32.527899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:32.527899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:32.529001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:14:32.529001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:14:32.529001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:14:32.529001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:14:32.529001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:14:32.529001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:14:32.529001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:32.529001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:32.529001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 347/682] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:32.529001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630762fa-79ef-4235-8d56-3ca28ad5a7e2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:32.649431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630762fa-79ef-4235-8d56-3ca28ad5a7e2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:32.673240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45671f2d-0bcf-42ed-9de6-a0a834ddade9 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:32.697317 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db0220-b474-46a0-b554-edbdce066e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:32.698352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db0220-b474-46a0-b554-edbdce066e3a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:32.698481 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db0220-b474-46a0-b554-edbdce066e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:32.698660 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30db0220-b474-46a0-b554-edbdce066e3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:32.703954 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:32.715480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30db0220-b474-46a0-b554-edbdce066e3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:32.723597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-30db0220-b474-46a0-b554-edbdce066e3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:14:32.723597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:32.723597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:32.723597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:32.723597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:32.723597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:32.723597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:32.723597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:32.723597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:32.723597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:32.723597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:32.723597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:32.723597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:32.723597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:32.723597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:32.723597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:32.724611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:32.724611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:32.724611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:32.724611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:32.724611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:32.724611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:32.724611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:32.724611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:32.724611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:32.724611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:32.724611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:32.724611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:32.724611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:32.724611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:32.724611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:32.724611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:32.725438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:32.725438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:32.725438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:32.725438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:32.725438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:32.725438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:32.725438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:32.725438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:32.725438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:32.725438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:32.725438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:14:32.725438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:32.728836 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 348/683] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:14:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:32.729208 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db0220-b474-46a0-b554-edbdce066e3a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:32.732423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa3015-5319-4b80-986e-09a9a8b4c3a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:32.733236 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa3015-5319-4b80-986e-09a9a8b4c3a2 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:32.733523 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa3015-5319-4b80-986e-09a9a8b4c3a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:32.734399 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa3015-5319-4b80-986e-09a9a8b4c3a2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:32.743528 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa3015-5319-4b80-986e-09a9a8b4c3a2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:32.744999 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aa3015-5319-4b80-986e-09a9a8b4c3a2 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:32.760404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-e4aa3015-5319-4b80-986e-09a9a8b4c3a2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 10:14:32.760404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:32.760404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:32.760404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:32.760404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:32.760404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:32.760404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:32.760404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:32.760404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:32.760404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:32.760404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:32.760404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:32.760404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:14:32.760404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:14:32.760404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:14:32.760404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:14:32.761378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:32.761378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:32.761378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:14:32.761378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:32.761378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:14:32.761378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:32.761378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:32.761378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:32.761378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:32.761378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:32.761378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:32.761378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:32.761378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:32.761378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:32.761378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:32.761378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:32.761378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:32.761378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:32.762297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:32.762297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:32.762297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:14:32.762297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:14:32.762297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:14:32.762297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:14:32.762297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:14:32.762297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:14:32.762297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 10:14:32.762297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:32.762297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 336/684] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:14:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:32.762884 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa3015-5319-4b80-986e-09a9a8b4c3a2 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:32.800281 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:32.858803 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30db0220-b474-46a0-b554-edbdce066e3a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:32.874222 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aa3015-5319-4b80-986e-09a9a8b4c3a2 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:32.899547 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff14bb8-908d-49e1-8428-4075d9810267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:32.900409 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff14bb8-908d-49e1-8428-4075d9810267 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:32.900770 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff14bb8-908d-49e1-8428-4075d9810267 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:32.901168 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff14bb8-908d-49e1-8428-4075d9810267 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:32.901989 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff14bb8-908d-49e1-8428-4075d9810267 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:32.902374 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff14bb8-908d-49e1-8428-4075d9810267 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:32.904184 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff14bb8-908d-49e1-8428-4075d9810267 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:32.911698 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30651166-38b8-4a75-baf5-3a078d0c4132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:32.912548 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30651166-38b8-4a75-baf5-3a078d0c4132 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:32.912972 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30651166-38b8-4a75-baf5-3a078d0c4132 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:32.913387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30651166-38b8-4a75-baf5-3a078d0c4132 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:32.914344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30651166-38b8-4a75-baf5-3a078d0c4132 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:32.914835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30651166-38b8-4a75-baf5-3a078d0c4132 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:32.916391 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30651166-38b8-4a75-baf5-3a078d0c4132 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:32.918111 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 349/685] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:14:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:32.919133 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff14bb8-908d-49e1-8428-4075d9810267 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:32.926457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 337/686] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:14:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:32.933453 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30651166-38b8-4a75-baf5-3a078d0c4132 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:32.983178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:33.034716 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:33.036608 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff14bb8-908d-49e1-8428-4075d9810267 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:33.041959 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30651166-38b8-4a75-baf5-3a078d0c4132 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:33.073989 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245bbd73-b426-4cfb-9684-72986dbc19a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:33.074810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bbd73-b426-4cfb-9684-72986dbc19a0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:33.075181 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bbd73-b426-4cfb-9684-72986dbc19a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:33.075554 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bbd73-b426-4cfb-9684-72986dbc19a0 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:33.076610 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245bbd73-b426-4cfb-9684-72986dbc19a0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:33.076968 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245bbd73-b426-4cfb-9684-72986dbc19a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:33.078591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245bbd73-b426-4cfb-9684-72986dbc19a0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:33.080698 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a71c86-b886-48c5-b4ef-7163d19cce59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:33.081534 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a71c86-b886-48c5-b4ef-7163d19cce59 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:33.081960 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a71c86-b886-48c5-b4ef-7163d19cce59 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:33.082441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a71c86-b886-48c5-b4ef-7163d19cce59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:33.083473 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a71c86-b886-48c5-b4ef-7163d19cce59 None admin] RBAC: Adding query filter params (role.id=99d8cd9cd87e472290bf0de2710b5857, scope.project.id=50902e0539434c82b505dc5009aeaf85, user.id=059b05007c0144f7a50dbd9f0ef41359) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:33.083964 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a71c86-b886-48c5-b4ef-7163d19cce59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:33.085574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a71c86-b886-48c5-b4ef-7163d19cce59 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:33.092394 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 338/687] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:14:32 2022] GET /identity/v3/role_assignments?user.id=059b05007c0144f7a50dbd9f0ef41359&scope.project.id=50902e0539434c82b505dc5009aeaf85&role.id=99d8cd9cd87e472290bf0de2710b5857 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:33.093465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a71c86-b886-48c5-b4ef-7163d19cce59 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:33.099659 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 350/688] 10.222.0.47 () {56 vars in 1135 bytes} [Thu Jun 30 10:14:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:33.105540 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245bbd73-b426-4cfb-9684-72986dbc19a0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:33.209067 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a71c86-b886-48c5-b4ef-7163d19cce59 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:33.221964 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245bbd73-b426-4cfb-9684-72986dbc19a0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:33.238676 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:33.248202 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c3269-2f18-46cd-91d3-73129ff344bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:33.249258 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c3269-2f18-46cd-91d3-73129ff344bf None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:33.249700 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c3269-2f18-46cd-91d3-73129ff344bf None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:33.250134 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c3269-2f18-46cd-91d3-73129ff344bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:33.261704 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4569c9-6274-43e4-bd7d-976a7b250ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:33.262441 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4569c9-6274-43e4-bd7d-976a7b250ec7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:33.262730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4569c9-6274-43e4-bd7d-976a7b250ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:33.263012 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4569c9-6274-43e4-bd7d-976a7b250ec7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:33.264803 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c3269-2f18-46cd-91d3-73129ff344bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:33.266592 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c3269-2f18-46cd-91d3-73129ff344bf None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:33.304574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-6c7c3269-2f18-46cd-91d3-73129ff344bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:33.304574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:33.304574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:33.304574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:33.304574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:33.304574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:33.304574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:33.304574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:33.304574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:33.304574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:33.304574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:33.304574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:33.304574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:33.304574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:33.304574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:33.305593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:33.305593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:33.305593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:33.305593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:33.305593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:33.305593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:33.305593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:33.305593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:33.305593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:33.305593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:33.305593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:33.305593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:33.305593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:33.305593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:33.305593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:33.305593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:33.305593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:33.307093 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:33.307093 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:33.307093 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:33.307093 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:33.310159 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 339/689] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:33.317261 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c3269-2f18-46cd-91d3-73129ff344bf None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:33.344747 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:33.350405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4569c9-6274-43e4-bd7d-976a7b250ec7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:33.352070 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4569c9-6274-43e4-bd7d-976a7b250ec7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:33.444954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c3269-2f18-46cd-91d3-73129ff344bf None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:33.465988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 351/690] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:14:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 30 10:14:33.472389 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4569c9-6274-43e4-bd7d-976a7b250ec7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:33.480077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f81d6-8287-4c85-9c98-7661c6550347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:33.480760 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f81d6-8287-4c85-9c98-7661c6550347 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:33.481044 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f81d6-8287-4c85-9c98-7661c6550347 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:33.481325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f81d6-8287-4c85-9c98-7661c6550347 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:33.482060 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77f81d6-8287-4c85-9c98-7661c6550347 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:33.482767 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77f81d6-8287-4c85-9c98-7661c6550347 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:33.483792 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77f81d6-8287-4c85-9c98-7661c6550347 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:33.491573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:33.497328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 340/691] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:14:33.503634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f81d6-8287-4c85-9c98-7661c6550347 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:33.582620 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:33.590088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4569c9-6274-43e4-bd7d-976a7b250ec7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:33.617238 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f81d6-8287-4c85-9c98-7661c6550347 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:33.631933 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2c720-4f2c-497b-8016-0923694be1e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:33.633118 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2c720-4f2c-497b-8016-0923694be1e0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:33.633661 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2c720-4f2c-497b-8016-0923694be1e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:33.634139 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d2c720-4f2c-497b-8016-0923694be1e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:33.645779 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2c720-4f2c-497b-8016-0923694be1e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:33.647837 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d2c720-4f2c-497b-8016-0923694be1e0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:33.665757 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10cb16-b3ec-4d85-87d3-b1e390a1ad48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:33.666567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10cb16-b3ec-4d85-87d3-b1e390a1ad48 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:33.666848 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10cb16-b3ec-4d85-87d3-b1e390a1ad48 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:33.667123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f10cb16-b3ec-4d85-87d3-b1e390a1ad48 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:33.667594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-48d2c720-4f2c-497b-8016-0923694be1e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:33.667594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:33.667594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:33.667594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:33.667594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:33.667594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:33.667594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:33.667594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:33.667594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:33.667594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:33.667594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:33.667594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:33.667594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:33.667594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:33.667594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:33.668562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:33.668562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:33.668562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:33.668562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:33.668562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:33.668562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:33.668562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:33.668562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:33.668562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:33.668562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:33.668562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:33.668562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:33.668562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:33.668562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:33.668562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:33.668562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:33.668562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:33.669490 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:33.669490 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:33.669490 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:33.669490 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:33.671707 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 352/692] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:14:33.677621 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2c720-4f2c-497b-8016-0923694be1e0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:33.692132 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10cb16-b3ec-4d85-87d3-b1e390a1ad48 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:33.693611 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f10cb16-b3ec-4d85-87d3-b1e390a1ad48 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:33.722892 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-7f10cb16-b3ec-4d85-87d3-b1e390a1ad48 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 10:14:33.722892 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:33.722892 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:33.722892 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:33.722892 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:33.722892 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:33.722892 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:33.722892 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:33.722892 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:33.722892 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:33.722892 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:33.722892 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:33.722892 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:14:33.722892 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:14:33.722892 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:14:33.722892 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:14:33.723961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:33.723961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:33.723961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:14:33.723961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:33.723961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:14:33.723961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:33.723961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:33.723961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:33.723961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:33.723961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:33.723961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:33.723961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:33.723961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:33.723961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:33.723961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:33.723961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:33.723961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:33.723961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:33.724930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:33.724930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:33.724930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:14:33.724930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:14:33.724930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:14:33.724930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:14:33.724930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:14:33.724930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:14:33.724930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 10:14:33.724930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:33.736659 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 341/693] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jun 30 10:14:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:33.744264 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10cb16-b3ec-4d85-87d3-b1e390a1ad48 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:33.754112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:33.787663 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d2c720-4f2c-497b-8016-0923694be1e0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:33.823171 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9785e-8244-4f1c-aad4-bb0950a3b6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:33.824423 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9785e-8244-4f1c-aad4-bb0950a3b6a8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:33.824912 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9785e-8244-4f1c-aad4-bb0950a3b6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:33.825415 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9785e-8244-4f1c-aad4-bb0950a3b6a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:33.826361 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e9785e-8244-4f1c-aad4-bb0950a3b6a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:33.826872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e9785e-8244-4f1c-aad4-bb0950a3b6a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:33.828607 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e9785e-8244-4f1c-aad4-bb0950a3b6a8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:33.841598 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 353/694] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:33.847379 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9785e-8244-4f1c-aad4-bb0950a3b6a8 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:33.955779 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9785e-8244-4f1c-aad4-bb0950a3b6a8 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:33.984121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:33.996785 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f10ba-9c31-4b21-ab26-b0834b428d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:33.996785 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f10ba-9c31-4b21-ab26-b0834b428d3d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:33.996785 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f10ba-9c31-4b21-ab26-b0834b428d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:33.996785 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f10ba-9c31-4b21-ab26-b0834b428d3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:34.004684 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f10ba-9c31-4b21-ab26-b0834b428d3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:34.006460 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8f10ba-9c31-4b21-ab26-b0834b428d3d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:34.027852 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-3b8f10ba-9c31-4b21-ab26-b0834b428d3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:34.027852 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:34.027852 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:34.027852 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:34.027852 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:34.027852 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:34.027852 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:34.027852 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:34.027852 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:34.027852 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:34.027852 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:34.027852 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:34.027852 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:14:34.027852 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:14:34.027852 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:14:34.028811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:14:34.028811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:34.028811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:34.028811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:34.028811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:34.028811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:34.028811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:34.028811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:34.028811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:34.028811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:34.028811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:34.028811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:34.028811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:34.028811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:34.028811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:34.028811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:34.028811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:34.029717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:14:34.029717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:14:34.029717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:14:34.029717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:14:34.029717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:14:34.029717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:14:34.029717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:34.029717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:34.031997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 354/695] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:14:34.040218 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f10cb16-b3ec-4d85-87d3-b1e390a1ad48 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:34.048899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f10ba-9c31-4b21-ab26-b0834b428d3d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:34.076594 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:34.084556 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ae7bc-0111-407d-b04c-fa8ed75fc890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:34.085522 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ae7bc-0111-407d-b04c-fa8ed75fc890 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:34.085805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ae7bc-0111-407d-b04c-fa8ed75fc890 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:34.088461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ae7bc-0111-407d-b04c-fa8ed75fc890 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:34.088461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ae7bc-0111-407d-b04c-fa8ed75fc890 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:34.088461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ae7bc-0111-407d-b04c-fa8ed75fc890 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:34.090238 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ae7bc-0111-407d-b04c-fa8ed75fc890 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:34.115011 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 342/696] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jun 30 10:14:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:34.116989 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef376fca-72d4-4a72-b8d4-f9a05177d600 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:34.117284 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef376fca-72d4-4a72-b8d4-f9a05177d600 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:34.117824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef376fca-72d4-4a72-b8d4-f9a05177d600 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:34.119703 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 343/697] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:34.120809 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef376fca-72d4-4a72-b8d4-f9a05177d600 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:34.173345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f10ba-9c31-4b21-ab26-b0834b428d3d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:34.211912 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87277966-c4f9-4eab-b46f-6bb5151a094b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:34.212652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87277966-c4f9-4eab-b46f-6bb5151a094b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:34.212979 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87277966-c4f9-4eab-b46f-6bb5151a094b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:34.213254 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87277966-c4f9-4eab-b46f-6bb5151a094b None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:34.214115 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87277966-c4f9-4eab-b46f-6bb5151a094b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:34.214423 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87277966-c4f9-4eab-b46f-6bb5151a094b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:34.216112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87277966-c4f9-4eab-b46f-6bb5151a094b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:34.235770 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 355/698] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:14:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:34.237282 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db919f7-d3f7-4332-a10e-6db6004a5e8f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:34.237576 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db919f7-d3f7-4332-a10e-6db6004a5e8f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:34.237859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db919f7-d3f7-4332-a10e-6db6004a5e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:34.256283 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4db919f7-d3f7-4332-a10e-6db6004a5e8f None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:34.257405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4db919f7-d3f7-4332-a10e-6db6004a5e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:34.257643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef376fca-72d4-4a72-b8d4-f9a05177d600 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:34.299587 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59036418-04ec-4679-8cb6-07a16810c35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:34.300247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59036418-04ec-4679-8cb6-07a16810c35b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:34.300591 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59036418-04ec-4679-8cb6-07a16810c35b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:34.300874 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59036418-04ec-4679-8cb6-07a16810c35b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:34.311541 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:34.347602 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 356/699] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:34.348702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db919f7-d3f7-4332-a10e-6db6004a5e8f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:34.366839 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:34.381664 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59036418-04ec-4679-8cb6-07a16810c35b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:34.383422 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59036418-04ec-4679-8cb6-07a16810c35b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:34.473893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db919f7-d3f7-4332-a10e-6db6004a5e8f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:34.497179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 344/700] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:14:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:14:34.498431 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59036418-04ec-4679-8cb6-07a16810c35b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:34.514759 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eeb4ac-373c-4b37-b8d9-4837747d51e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:34.515682 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eeb4ac-373c-4b37-b8d9-4837747d51e3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:34.516067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eeb4ac-373c-4b37-b8d9-4837747d51e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:34.516685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eeb4ac-373c-4b37-b8d9-4837747d51e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:34.518365 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eeb4ac-373c-4b37-b8d9-4837747d51e3 None admin] RBAC: Adding query filter params (role.id=99d8cd9cd87e472290bf0de2710b5857, scope.project.id=50902e0539434c82b505dc5009aeaf85, user.id=07de926cb8594b098f79111dab8a9cc6) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:34.518365 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eeb4ac-373c-4b37-b8d9-4837747d51e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:34.520064 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eeb4ac-373c-4b37-b8d9-4837747d51e3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:34.531665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 357/701] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:14:34 2022] GET /identity/v3/role_assignments?user.id=07de926cb8594b098f79111dab8a9cc6&scope.project.id=50902e0539434c82b505dc5009aeaf85&role.id=99d8cd9cd87e472290bf0de2710b5857 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:34.532815 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eeb4ac-373c-4b37-b8d9-4837747d51e3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:34.611693 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:34.617361 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59036418-04ec-4679-8cb6-07a16810c35b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:34.642924 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:34.659231 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98547366-6406-46e2-b890-c02d23d68148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:34.660474 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98547366-6406-46e2-b890-c02d23d68148 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:34.660964 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98547366-6406-46e2-b890-c02d23d68148 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:34.661432 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98547366-6406-46e2-b890-c02d23d68148 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:34.673061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98547366-6406-46e2-b890-c02d23d68148 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:34.675646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98547366-6406-46e2-b890-c02d23d68148 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:34.698780 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-98547366-6406-46e2-b890-c02d23d68148 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:34.698780 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:34.698780 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:34.698780 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:34.698780 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:34.698780 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:34.698780 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:34.698780 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:34.698780 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:34.698780 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:34.698780 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:34.698780 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:34.698780 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:34.698780 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:34.698780 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:34.699769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:34.699769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:34.699769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:34.699769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:34.699769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:34.699769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:34.699769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:34.699769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:34.699769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:34.699769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:34.699769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:34.699769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:34.699769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:34.699769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:34.699769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:34.699769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:34.699769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:34.700771 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:34.700771 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:34.700771 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:34.700771 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:34.703406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 345/702] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:34.710331 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98547366-6406-46e2-b890-c02d23d68148 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:34.768626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eeb4ac-373c-4b37-b8d9-4837747d51e3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:34.816186 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43b85e-14db-4016-ac2c-46ca3c8e905f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:34.817253 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43b85e-14db-4016-ac2c-46ca3c8e905f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:34.818054 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43b85e-14db-4016-ac2c-46ca3c8e905f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:34.818479 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d43b85e-14db-4016-ac2c-46ca3c8e905f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:34.836074 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d43b85e-14db-4016-ac2c-46ca3c8e905f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:34.838192 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d43b85e-14db-4016-ac2c-46ca3c8e905f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:34.846399 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98547366-6406-46e2-b890-c02d23d68148 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:34.860153 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-8d43b85e-14db-4016-ac2c-46ca3c8e905f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:34.860153 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:34.860153 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:34.860153 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:34.860153 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:34.860153 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:34.860153 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:34.860153 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:34.860153 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:34.860153 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:34.860153 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:34.860153 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:34.860153 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:34.860153 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:34.860153 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:34.861776 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:34.861776 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:34.861776 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:34.861776 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:34.861776 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:34.861776 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:34.861776 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:34.861776 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:34.861776 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:34.861776 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:34.861776 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:34.861776 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:34.861776 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:34.861776 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:34.861776 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:34.861776 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:34.861776 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:34.862707 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:34.862707 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:34.862707 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:34.862707 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:34.865625 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 358/703] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:34.870681 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43b85e-14db-4016-ac2c-46ca3c8e905f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:34.889179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d82d5-1d37-4d91-a36c-097bb328352b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:34.889998 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d82d5-1d37-4d91-a36c-097bb328352b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:34.890419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d82d5-1d37-4d91-a36c-097bb328352b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:34.890785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d82d5-1d37-4d91-a36c-097bb328352b None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:34.891773 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d82d5-1d37-4d91-a36c-097bb328352b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:34.892751 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d82d5-1d37-4d91-a36c-097bb328352b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:34.894369 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d82d5-1d37-4d91-a36c-097bb328352b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:34.904916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:34.911160 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 346/704] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:34.921283 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d82d5-1d37-4d91-a36c-097bb328352b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:35.000882 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:35.012285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d43b85e-14db-4016-ac2c-46ca3c8e905f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:35.045670 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d82d5-1d37-4d91-a36c-097bb328352b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:35.058665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ed7d71-2f33-413d-b132-cd320e0892cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:35.059607 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed7d71-2f33-413d-b132-cd320e0892cf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:35.060030 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed7d71-2f33-413d-b132-cd320e0892cf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:35.060462 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ed7d71-2f33-413d-b132-cd320e0892cf None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:35.061198 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed7d71-2f33-413d-b132-cd320e0892cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:35.061508 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed7d71-2f33-413d-b132-cd320e0892cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:35.062889 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ed7d71-2f33-413d-b132-cd320e0892cf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:35.075359 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 359/705] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:35.076682 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9eb28a-afc1-4ddb-9ace-83159ad4c06b None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:35.076980 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9eb28a-afc1-4ddb-9ace-83159ad4c06b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:35.083290 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9eb28a-afc1-4ddb-9ace-83159ad4c06b None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:35.083290 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 360/706] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:35.083290 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9eb28a-afc1-4ddb-9ace-83159ad4c06b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:35.101296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f332b68d-d3f7-499e-9faf-66e8b380b078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:35.102383 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332b68d-d3f7-499e-9faf-66e8b380b078 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:35.102876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332b68d-d3f7-499e-9faf-66e8b380b078 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:35.103519 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f332b68d-d3f7-499e-9faf-66e8b380b078 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:35.104553 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f332b68d-d3f7-499e-9faf-66e8b380b078 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:35.106294 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f332b68d-d3f7-499e-9faf-66e8b380b078 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:35.195613 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 347/707] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jun 30 10:14:34 2022] POST /identity/v3/users => generated 254 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:14:35.198038 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9eb28a-afc1-4ddb-9ace-83159ad4c06b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:35.198587 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84589879-6fef-4a0d-aab7-ca7a583bec25 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:35.199071 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84589879-6fef-4a0d-aab7-ca7a583bec25 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:35.199563 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84589879-6fef-4a0d-aab7-ca7a583bec25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:35.238090 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-84589879-6fef-4a0d-aab7-ca7a583bec25 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:35.239858 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-84589879-6fef-4a0d-aab7-ca7a583bec25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:35.250184 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6cac71-c20e-48b1-b924-558a051cc409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:35.252664 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cac71-c20e-48b1-b924-558a051cc409 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:35.253004 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cac71-c20e-48b1-b924-558a051cc409 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:35.253384 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cac71-c20e-48b1-b924-558a051cc409 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:35.262509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6cac71-c20e-48b1-b924-558a051cc409 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:35.264126 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6cac71-c20e-48b1-b924-558a051cc409 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:35.283474 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-4f6cac71-c20e-48b1-b924-558a051cc409 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:35.283474 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:35.283474 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:35.283474 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:35.283474 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:35.283474 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:35.283474 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:35.283474 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:35.283474 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:35.283474 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:35.283474 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:35.283474 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:35.283474 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:14:35.283474 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:14:35.283474 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:14:35.284863 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:14:35.284863 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:35.284863 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:35.284863 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:35.284863 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:35.284863 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:35.284863 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:35.284863 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:35.284863 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:35.284863 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:35.284863 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:35.284863 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:35.284863 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:35.284863 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:35.284863 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:35.284863 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:35.284863 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:35.285734 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:14:35.285734 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:14:35.285734 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:14:35.285734 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:14:35.285734 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:14:35.285734 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:14:35.285734 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:35.285734 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:35.285734 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 361/708] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:35.304875 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6cac71-c20e-48b1-b924-558a051cc409 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:35.342766 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:35.351643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:35.374677 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 348/709] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:35.415708 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84589879-6fef-4a0d-aab7-ca7a583bec25 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:35.523998 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6cac71-c20e-48b1-b924-558a051cc409 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:35.551689 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84589879-6fef-4a0d-aab7-ca7a583bec25 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:35.575001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b519235-9de4-4436-83f7-ba79d1e4b39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:35.575702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b519235-9de4-4436-83f7-ba79d1e4b39a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:35.575986 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b519235-9de4-4436-83f7-ba79d1e4b39a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:35.576261 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b519235-9de4-4436-83f7-ba79d1e4b39a None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:35.577186 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b519235-9de4-4436-83f7-ba79d1e4b39a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:35.577502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b519235-9de4-4436-83f7-ba79d1e4b39a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:35.578940 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b519235-9de4-4436-83f7-ba79d1e4b39a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:35.595306 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 362/710] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:14:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:35.598649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b519235-9de4-4436-83f7-ba79d1e4b39a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:35.617807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a9587-6ea2-477c-8958-13499f4e7166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:35.618637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a9587-6ea2-477c-8958-13499f4e7166 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:35.618919 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a9587-6ea2-477c-8958-13499f4e7166 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:35.619194 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5a9587-6ea2-477c-8958-13499f4e7166 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:35.619904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a9587-6ea2-477c-8958-13499f4e7166 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:35.621339 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5a9587-6ea2-477c-8958-13499f4e7166 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:35.637739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-6c5a9587-6ea2-477c-8958-13499f4e7166 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 10:14:35.637739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:35.637739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 10:14:35.637739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 10:14:35.637739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 10:14:35.637739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 10:14:35.637739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 10:14:35.637739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:14:35.637739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 10:14:35.637739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:35.637739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:14:35.637739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:35.637739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:35.637739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:35.637739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:35.637739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:35.638784 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:35.638784 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:35.638784 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:35.638784 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:35.638784 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:35.638784 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:35.638784 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:35.638784 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 10:14:35.638784 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 10:14:35.638784 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 10:14:35.638784 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 10:14:35.638784 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:35.638784 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:35.638784 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:35.638784 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:35.638784 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:35.640312 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:35.640312 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:35.640312 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:35.640312 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:35.640312 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:35.640312 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:35.640312 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:35.640312 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:35.640312 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:35.640312 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:35.640312 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:35.640312 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 10:14:35.640312 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:14:35.640312 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 10:14:35.640312 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:35.640312 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 349/711] 10.222.0.47 () {56 vars in 1129 bytes} [Thu Jun 30 10:14:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:14:35.646404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a9587-6ea2-477c-8958-13499f4e7166 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:35.686998 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:35.705578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:35.721871 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b519235-9de4-4436-83f7-ba79d1e4b39a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:35.758212 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917b3aa9-d8af-477b-bcfd-81358edd7466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:35.759175 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b3aa9-d8af-477b-bcfd-81358edd7466 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:35.759680 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b3aa9-d8af-477b-bcfd-81358edd7466 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:35.760043 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917b3aa9-d8af-477b-bcfd-81358edd7466 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:35.760892 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b3aa9-d8af-477b-bcfd-81358edd7466 None admin] RBAC: Adding query filter params (role.id=e63bda042e104a4a93fba15ffcfe1183, scope.project.id=50902e0539434c82b505dc5009aeaf85, user.id=71f6d7715e6f49ffb9fdf6e1307bbd51) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:35.761210 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b3aa9-d8af-477b-bcfd-81358edd7466 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:35.762652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917b3aa9-d8af-477b-bcfd-81358edd7466 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:35.763237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5a9587-6ea2-477c-8958-13499f4e7166 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:35.776122 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 363/712] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:14:35 2022] GET /identity/v3/role_assignments?user.id=71f6d7715e6f49ffb9fdf6e1307bbd51&scope.project.id=50902e0539434c82b505dc5009aeaf85&role.id=e63bda042e104a4a93fba15ffcfe1183 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:35.777408 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f1d33-1425-4494-bea6-41f6f9b0371f None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:35.777702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f1d33-1425-4494-bea6-41f6f9b0371f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:35.777985 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f1d33-1425-4494-bea6-41f6f9b0371f None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:35.779694 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 364/713] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:35.785588 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bfffd-5020-4dca-ab86-4c0349c3b82c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:35.785921 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bfffd-5020-4dca-ab86-4c0349c3b82c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:35.786156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8bfffd-5020-4dca-ab86-4c0349c3b82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:35.819224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1e8bfffd-5020-4dca-ab86-4c0349c3b82c None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:35.820490 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1e8bfffd-5020-4dca-ab86-4c0349c3b82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:35.823853 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0aa1bf-a1a4-4c83-9f7d-30d2b317d9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:35.824548 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0aa1bf-a1a4-4c83-9f7d-30d2b317d9d1 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:35.824839 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0aa1bf-a1a4-4c83-9f7d-30d2b317d9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:35.825161 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0aa1bf-a1a4-4c83-9f7d-30d2b317d9d1 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:35.825834 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0aa1bf-a1a4-4c83-9f7d-30d2b317d9d1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:35.826142 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0aa1bf-a1a4-4c83-9f7d-30d2b317d9d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:35.827620 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0aa1bf-a1a4-4c83-9f7d-30d2b317d9d1 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:35.838795 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 350/714] 10.222.0.47 () {56 vars in 1124 bytes} [Thu Jun 30 10:14:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:35.844296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0aa1bf-a1a4-4c83-9f7d-30d2b317d9d1 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:35.959969 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 365/715] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 10:14:35.994112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8bfffd-5020-4dca-ab86-4c0349c3b82c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:36.021321 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0aa1bf-a1a4-4c83-9f7d-30d2b317d9d1 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:36.048545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:36.076565 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:36.100539 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b3c1d-ea05-468d-8c5c-206b6eb18cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:36.101668 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b3c1d-ea05-468d-8c5c-206b6eb18cf3 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:36.102268 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b3c1d-ea05-468d-8c5c-206b6eb18cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:36.102745 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b3c1d-ea05-468d-8c5c-206b6eb18cf3 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:36.103687 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b3c1d-ea05-468d-8c5c-206b6eb18cf3 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:36.104198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b3c1d-ea05-468d-8c5c-206b6eb18cf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:36.105808 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b3c1d-ea05-468d-8c5c-206b6eb18cf3 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:36.118105 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8bfffd-5020-4dca-ab86-4c0349c3b82c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:36.121480 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 351/716] 10.222.0.47 () {56 vars in 1124 bytes} [Thu Jun 30 10:14:35 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:36.158945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19703c4-e55b-41a1-9a42-119294f485f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:36.159897 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19703c4-e55b-41a1-9a42-119294f485f9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:36.160185 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19703c4-e55b-41a1-9a42-119294f485f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:36.160466 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19703c4-e55b-41a1-9a42-119294f485f9 None admin] PATH_INFO: `/v3/services/network` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:36.161138 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19703c4-e55b-41a1-9a42-119294f485f9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:36.162535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19703c4-e55b-41a1-9a42-119294f485f9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:36.177164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-f19703c4-e55b-41a1-9a42-119294f485f9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 10:14:36.177164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:36.177164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 10:14:36.177164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 10:14:36.177164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 10:14:36.177164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 10:14:36.177164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 10:14:36.177164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:14:36.177164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 10:14:36.177164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:36.177164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:14:36.177164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:36.177164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:36.177164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:36.177164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:36.177164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:36.177164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:36.178269 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:36.178269 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:36.178269 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:36.178269 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:36.178269 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:36.178269 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:36.178269 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 10:14:36.178269 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 10:14:36.178269 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 10:14:36.178269 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 10:14:36.178269 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:36.178269 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:36.178269 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:36.178269 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:36.178269 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:36.178269 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:36.179183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:36.179183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:36.179183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:36.179183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:36.179183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:36.179183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:36.179183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:36.179183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:36.179183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:36.179183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:36.179183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 10:14:36.179183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:14:36.179183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 10:14:36.179183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:36.179183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 366/717] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:35 2022] GET /identity/v3/services/network => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:14:36.184632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b3c1d-ea05-468d-8c5c-206b6eb18cf3 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:36.342512 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b3c1d-ea05-468d-8c5c-206b6eb18cf3 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:36.379185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe22a5f-df47-493f-97a8-ed0957b0d059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:36.380712 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe22a5f-df47-493f-97a8-ed0957b0d059 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:36.381238 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe22a5f-df47-493f-97a8-ed0957b0d059 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:36.381716 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe22a5f-df47-493f-97a8-ed0957b0d059 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:36.382623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe22a5f-df47-493f-97a8-ed0957b0d059 None admin] RBAC: Adding query filter params (name=network) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:36.383156 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe22a5f-df47-493f-97a8-ed0957b0d059 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:36.384800 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe22a5f-df47-493f-97a8-ed0957b0d059 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:36.389981 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 352/718] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jun 30 10:14:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:36.395106 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19703c4-e55b-41a1-9a42-119294f485f9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:36.518997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19703c4-e55b-41a1-9a42-119294f485f9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:36.573667 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:36.585359 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e24618-875a-4c5d-8bb5-591b8d07d63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:36.585359 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e24618-875a-4c5d-8bb5-591b8d07d63f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:36.585359 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e24618-875a-4c5d-8bb5-591b8d07d63f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:36.585359 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e24618-875a-4c5d-8bb5-591b8d07d63f None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:36.594093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e24618-875a-4c5d-8bb5-591b8d07d63f None admin] RBAC: Adding query filter params (type=network) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:36.594093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e24618-875a-4c5d-8bb5-591b8d07d63f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:36.594093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e24618-875a-4c5d-8bb5-591b8d07d63f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:36.599624 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 367/719] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jun 30 10:14:36 2022] GET /identity/v3/services?type=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:14:37.350634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af5185-86de-4a81-b2c3-8b1210d9ca38 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:37.351046 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af5185-86de-4a81-b2c3-8b1210d9ca38 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:37.351182 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af5185-86de-4a81-b2c3-8b1210d9ca38 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:37.352785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 353/720] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:37.358134 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095deb62-fab8-4554-8be1-ef7e641e9adb None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:37.358685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095deb62-fab8-4554-8be1-ef7e641e9adb None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:37.359199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095deb62-fab8-4554-8be1-ef7e641e9adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:37.392666 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-095deb62-fab8-4554-8be1-ef7e641e9adb None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:37.394225 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-095deb62-fab8-4554-8be1-ef7e641e9adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:37.474529 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 368/721] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:37.529911 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af5185-86de-4a81-b2c3-8b1210d9ca38 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:37.591448 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:37.656817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af5185-86de-4a81-b2c3-8b1210d9ca38 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:37.692455 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe12aa4-965f-4dd3-a1d0-cac327a4c99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:37.693259 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe12aa4-965f-4dd3-a1d0-cac327a4c99b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:37.693551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe12aa4-965f-4dd3-a1d0-cac327a4c99b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:37.693832 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe12aa4-965f-4dd3-a1d0-cac327a4c99b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:37.704558 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe12aa4-965f-4dd3-a1d0-cac327a4c99b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:37.713130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-bbe12aa4-965f-4dd3-a1d0-cac327a4c99b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:14:37.713130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:37.713130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:37.713130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:37.713130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:37.713130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:37.713130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:37.713130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:37.713130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:37.713130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:37.713130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:37.713130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:37.713130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:37.713130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:37.713130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:37.713130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:37.714112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:37.714112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:37.714112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:37.714112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:37.714112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:37.714112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:37.714112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:37.714112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:37.714112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:37.714112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:37.714112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:37.714112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:37.714112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:37.714112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:37.714112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:37.714112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:37.714939 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:37.714939 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:37.714939 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:37.714939 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:37.714939 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:37.714939 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:37.714939 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:37.714939 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:37.714939 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:37.714939 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:37.714939 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:14:37.714939 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:37.714939 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 354/722] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:14:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:37.724395 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095deb62-fab8-4554-8be1-ef7e641e9adb None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:37.888127 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095deb62-fab8-4554-8be1-ef7e641e9adb None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:37.927226 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:37.943975 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5bab26-f0dc-42b8-bac0-3d08041d8ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:37.944665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5bab26-f0dc-42b8-bac0-3d08041d8ade None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:37.944952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5bab26-f0dc-42b8-bac0-3d08041d8ade None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:37.945230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5bab26-f0dc-42b8-bac0-3d08041d8ade None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:37.946051 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5bab26-f0dc-42b8-bac0-3d08041d8ade None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:37.946397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5bab26-f0dc-42b8-bac0-3d08041d8ade None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:37.947890 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5bab26-f0dc-42b8-bac0-3d08041d8ade None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:37.956747 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 369/723] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:14:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:37.962266 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe12aa4-965f-4dd3-a1d0-cac327a4c99b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:38.176953 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe12aa4-965f-4dd3-a1d0-cac327a4c99b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:38.179637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:38.309006 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa833d04-4586-44a4-9ed9-ebcb2f21646a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:38.309822 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa833d04-4586-44a4-9ed9-ebcb2f21646a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:38.310110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa833d04-4586-44a4-9ed9-ebcb2f21646a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:38.310384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa833d04-4586-44a4-9ed9-ebcb2f21646a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:38.329254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa833d04-4586-44a4-9ed9-ebcb2f21646a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:38.331325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa833d04-4586-44a4-9ed9-ebcb2f21646a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:38.336895 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b3747-8d30-47d2-bc15-a890579eb786 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:38.337190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b3747-8d30-47d2-bc15-a890579eb786 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:38.337474 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587b3747-8d30-47d2-bc15-a890579eb786 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:38.339119 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 370/724] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:38.352318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-fa833d04-4586-44a4-9ed9-ebcb2f21646a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:38.352318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:38.352318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:38.352318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:38.352318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:38.352318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:38.352318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:38.352318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:38.352318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:38.352318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:38.352318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:38.352318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:38.352318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:38.352318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:38.352318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:38.353230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:38.353230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:38.353230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:38.353230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:38.353230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:38.353230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:38.353230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:38.353230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:38.353230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:38.353230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:38.353230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:38.353230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:38.353230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:38.353230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:38.353230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:38.353230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:38.353230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:38.354190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:38.354190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:38.354190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:38.354190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:38.354409 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23475a-314d-460d-9846-184b2244613d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:38.354409 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23475a-314d-460d-9846-184b2244613d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:38.354409 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23475a-314d-460d-9846-184b2244613d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:38.354595 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 355/725] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:38.359898 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa833d04-4586-44a4-9ed9-ebcb2f21646a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:38.404313 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0e23475a-314d-460d-9846-184b2244613d None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:38.405541 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0e23475a-314d-460d-9846-184b2244613d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:38.480097 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 371/726] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 10:14:38.481486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a990c-92b3-4a94-944d-628c904b233d None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:38.481782 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a990c-92b3-4a94-944d-628c904b233d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:38.481898 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa833d04-4586-44a4-9ed9-ebcb2f21646a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:38.482060 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62a990c-92b3-4a94-944d-628c904b233d None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:38.483348 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 372/727] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:38.489452 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464a882-34a9-4036-a911-b3df7bed91b4 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:38.489811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464a882-34a9-4036-a911-b3df7bed91b4 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:38.490171 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464a882-34a9-4036-a911-b3df7bed91b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:38.507713 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6464a882-34a9-4036-a911-b3df7bed91b4 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:38.508994 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6464a882-34a9-4036-a911-b3df7bed91b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:38.527797 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffb158-804d-43da-bf91-9524cb038a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:38.528476 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffb158-804d-43da-bf91-9524cb038a2e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:38.529005 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffb158-804d-43da-bf91-9524cb038a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:38.529293 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ffb158-804d-43da-bf91-9524cb038a2e None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:38.530412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ffb158-804d-43da-bf91-9524cb038a2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:38.531015 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ffb158-804d-43da-bf91-9524cb038a2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:38.532126 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ffb158-804d-43da-bf91-9524cb038a2e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:38.550349 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 356/728] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:38.551204 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffb158-804d-43da-bf91-9524cb038a2e None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:38.567506 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:38.594810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 373/729] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 30 10:14:38.596274 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464a882-34a9-4036-a911-b3df7bed91b4 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:38.654459 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:38.681972 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ffb158-804d-43da-bf91-9524cb038a2e None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:38.724858 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab80ad2-65ae-487f-9e4c-8b45c27be018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:38.725773 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab80ad2-65ae-487f-9e4c-8b45c27be018 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:38.726131 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab80ad2-65ae-487f-9e4c-8b45c27be018 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:38.726474 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab80ad2-65ae-487f-9e4c-8b45c27be018 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:38.728661 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464a882-34a9-4036-a911-b3df7bed91b4 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:38.739579 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab80ad2-65ae-487f-9e4c-8b45c27be018 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:38.747412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-bab80ad2-65ae-487f-9e4c-8b45c27be018 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:14:38.747412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:38.747412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:38.747412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:38.747412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:38.747412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:38.747412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:38.747412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:38.747412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:38.747412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:38.747412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:38.747412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:38.747412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:38.747412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:38.747412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:38.747412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:38.749245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:38.749245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:38.749245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:38.749245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:38.749245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:38.749245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:38.749245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:38.749245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:38.749245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:38.749245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:38.749245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:38.749245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:38.749245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:38.749245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:38.749245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:38.749245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:38.751211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:38.751211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:38.751211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:38.751211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:38.751211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:38.751211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:38.751211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:38.751211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:38.751211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:38.751211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:38.751211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:14:38.751211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:38.751211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 357/730] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:14:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:38.751211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab80ad2-65ae-487f-9e4c-8b45c27be018 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:38.768192 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93639b55-a915-4b7d-92d0-3e067800bdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:38.769047 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93639b55-a915-4b7d-92d0-3e067800bdf9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:38.769335 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93639b55-a915-4b7d-92d0-3e067800bdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:38.769617 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93639b55-a915-4b7d-92d0-3e067800bdf9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:38.777886 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93639b55-a915-4b7d-92d0-3e067800bdf9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:38.779504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93639b55-a915-4b7d-92d0-3e067800bdf9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:38.795413 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-93639b55-a915-4b7d-92d0-3e067800bdf9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 10:14:38.795413 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:38.795413 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:38.795413 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:38.795413 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:38.795413 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:38.795413 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:38.795413 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:38.795413 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:38.795413 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:38.795413 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:38.795413 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:38.795413 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:14:38.795413 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:14:38.795413 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:14:38.795413 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:14:38.797535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:38.797535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:38.797535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:14:38.797535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:38.797535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:14:38.797535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:38.797535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:38.797535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:38.797535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:38.797535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:38.797535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:38.797535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:38.797535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:38.797535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:38.797535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:38.797535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:38.797535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:38.797535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:38.800099 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:38.800099 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:38.800099 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:14:38.800099 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:14:38.800099 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:14:38.800099 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:14:38.800099 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:14:38.800099 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:14:38.800099 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 10:14:38.800099 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:38.800099 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 374/731] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:14:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:38.800099 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93639b55-a915-4b7d-92d0-3e067800bdf9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:38.851031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:38.880596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab80ad2-65ae-487f-9e4c-8b45c27be018 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:38.907329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93639b55-a915-4b7d-92d0-3e067800bdf9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:38.914982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:38.919301 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f4d86-81fb-4007-8f31-2cf4e4f2de88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:38.920085 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f4d86-81fb-4007-8f31-2cf4e4f2de88 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:38.920370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f4d86-81fb-4007-8f31-2cf4e4f2de88 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:38.920644 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f4d86-81fb-4007-8f31-2cf4e4f2de88 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:38.932109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f4d86-81fb-4007-8f31-2cf4e4f2de88 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:38.940880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-1a5f4d86-81fb-4007-8f31-2cf4e4f2de88 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:38.940880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:38.940880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:38.940880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:38.940880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:38.940880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:38.940880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:38.940880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:38.940880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:38.940880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:38.940880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:38.940880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:38.940880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:38.940880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:38.940880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:38.940880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:38.941810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:38.941810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:38.941810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:38.941810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:38.941810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:38.941810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:38.941810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:38.941810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:38.941810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:38.941810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:38.941810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:38.941810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:38.941810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:38.941810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:38.941810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:38.941810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:38.942681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:38.942681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:38.942681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:38.942681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:38.942681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:38.942681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:38.942681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:38.942681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:38.942681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:38.942681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:38.942681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:38.942681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:38.942681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 358/732] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jun 30 10:14:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:38.944241 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f4d86-81fb-4007-8f31-2cf4e4f2de88 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:38.947439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8521fc-7ae8-4c0c-8922-a088894b134a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:38.948129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8521fc-7ae8-4c0c-8922-a088894b134a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:38.948411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8521fc-7ae8-4c0c-8922-a088894b134a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:38.948685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8521fc-7ae8-4c0c-8922-a088894b134a None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:38.949443 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8521fc-7ae8-4c0c-8922-a088894b134a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:38.949747 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8521fc-7ae8-4c0c-8922-a088894b134a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:38.951116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8521fc-7ae8-4c0c-8922-a088894b134a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:38.965235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 375/733] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:14:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:38.965235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76886c-3830-4fb6-9930-564e381dd356 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:38.965235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76886c-3830-4fb6-9930-564e381dd356 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:38.965235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed76886c-3830-4fb6-9930-564e381dd356 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:38.967620 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 376/734] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:38.968385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76886c-3830-4fb6-9930-564e381dd356 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:39.057173 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f4d86-81fb-4007-8f31-2cf4e4f2de88 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:39.072745 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed76886c-3830-4fb6-9930-564e381dd356 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:39.093569 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863fc43-4e7a-44a5-9773-d04a4ec79d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:39.094244 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863fc43-4e7a-44a5-9773-d04a4ec79d39 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:39.094637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863fc43-4e7a-44a5-9773-d04a4ec79d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:39.094913 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9863fc43-4e7a-44a5-9773-d04a4ec79d39 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:39.095935 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863fc43-4e7a-44a5-9773-d04a4ec79d39 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:39.096247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863fc43-4e7a-44a5-9773-d04a4ec79d39 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:39.097794 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9863fc43-4e7a-44a5-9773-d04a4ec79d39 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:39.099582 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:39.109662 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc1b55e-55a6-4461-8506-8a4000de4b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:39.110332 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1b55e-55a6-4461-8506-8a4000de4b43 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:39.110646 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1b55e-55a6-4461-8506-8a4000de4b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:39.111216 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc1b55e-55a6-4461-8506-8a4000de4b43 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:39.112077 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1b55e-55a6-4461-8506-8a4000de4b43 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:39.112390 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1b55e-55a6-4461-8506-8a4000de4b43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:39.113772 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc1b55e-55a6-4461-8506-8a4000de4b43 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:39.121723 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 359/735] 10.222.0.47 () {56 vars in 1135 bytes} [Thu Jun 30 10:14:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:39.127136 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 377/736] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jun 30 10:14:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:39.127368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863fc43-4e7a-44a5-9773-d04a4ec79d39 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:39.127627 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae2ff5-8527-40e2-a17f-6f00a52eaf58 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:39.127627 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae2ff5-8527-40e2-a17f-6f00a52eaf58 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:39.127627 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae2ff5-8527-40e2-a17f-6f00a52eaf58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:39.141806 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-99ae2ff5-8527-40e2-a17f-6f00a52eaf58 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:39.143610 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-99ae2ff5-8527-40e2-a17f-6f00a52eaf58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:39.169978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:39.216764 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 378/737] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:39.218029 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae2ff5-8527-40e2-a17f-6f00a52eaf58 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:39.246018 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9863fc43-4e7a-44a5-9773-d04a4ec79d39 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:39.280759 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f662e-0204-48dc-8f02-4bc456a7999e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:39.281560 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f662e-0204-48dc-8f02-4bc456a7999e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:39.281839 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f662e-0204-48dc-8f02-4bc456a7999e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:39.282127 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f662e-0204-48dc-8f02-4bc456a7999e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:39.292556 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f662e-0204-48dc-8f02-4bc456a7999e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:39.293991 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f662e-0204-48dc-8f02-4bc456a7999e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:39.311305 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-f35f662e-0204-48dc-8f02-4bc456a7999e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:39.311305 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:39.311305 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:39.311305 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:39.311305 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:39.311305 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:39.311305 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:39.311305 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:39.311305 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:39.311305 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:39.311305 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:39.311305 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:39.311305 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:39.311305 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:39.311305 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:39.312380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:39.312380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:39.312380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:39.312380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:39.312380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:39.312380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:39.312380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:39.312380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:39.312380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:39.312380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:39.312380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:39.312380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:39.312380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:39.312380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:39.312380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:39.312380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:39.312380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:39.313435 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:39.313435 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:39.313435 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:39.313435 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:39.313435 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 360/738] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:39.313435 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f662e-0204-48dc-8f02-4bc456a7999e None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:39.327420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae2ff5-8527-40e2-a17f-6f00a52eaf58 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:39.359014 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:39.361762 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ba951-efc3-4035-8b28-b177f3d4fd0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:39.362488 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ba951-efc3-4035-8b28-b177f3d4fd0f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:39.362845 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ba951-efc3-4035-8b28-b177f3d4fd0f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:39.363193 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ba951-efc3-4035-8b28-b177f3d4fd0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:39.390280 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:39.413042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f662e-0204-48dc-8f02-4bc456a7999e None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:39.429806 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ba951-efc3-4035-8b28-b177f3d4fd0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:39.431322 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ba951-efc3-4035-8b28-b177f3d4fd0f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:39.446282 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfb912f-25cb-4bfc-8655-3617571e88ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:39.447074 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb912f-25cb-4bfc-8655-3617571e88ca None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:39.447501 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb912f-25cb-4bfc-8655-3617571e88ca None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:39.447780 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfb912f-25cb-4bfc-8655-3617571e88ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:39.458084 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb912f-25cb-4bfc-8655-3617571e88ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:39.459565 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfb912f-25cb-4bfc-8655-3617571e88ca None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:39.479247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-ccfb912f-25cb-4bfc-8655-3617571e88ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:39.479247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:39.479247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:39.479247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:39.479247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:39.479247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:39.479247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:39.479247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:39.479247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:39.479247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:39.479247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:39.479247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:39.479247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:39.479247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:39.479247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:39.480207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:39.480207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:39.480207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:39.480207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:39.480207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:39.480207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:39.480207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:39.480207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:39.480207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:39.480207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:39.480207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:39.480207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:39.480207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:39.480207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:39.480207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:39.480207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:39.480207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:39.481100 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:39.481100 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:39.481100 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:39.481100 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:39.481100 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 361/739] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:39.481824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c9aec-1c48-48f9-bd81-e23524e265cd None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:39.482098 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c9aec-1c48-48f9-bd81-e23524e265cd None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:39.482370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4c9aec-1c48-48f9-bd81-e23524e265cd None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:39.483657 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 362/740] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:39.487256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c9aec-1c48-48f9-bd81-e23524e265cd None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:39.539394 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 379/741] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:14:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 30 10:14:39.540248 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ba951-efc3-4035-8b28-b177f3d4fd0f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:39.581168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:39.587465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4c9aec-1c48-48f9-bd81-e23524e265cd None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:39.598430 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:39.621504 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c11bb0-0b34-43fd-8a21-79cd356f8c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:39.622167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c11bb0-0b34-43fd-8a21-79cd356f8c28 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:39.622453 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c11bb0-0b34-43fd-8a21-79cd356f8c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:39.622729 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c11bb0-0b34-43fd-8a21-79cd356f8c28 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:39.623473 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c11bb0-0b34-43fd-8a21-79cd356f8c28 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:39.624865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c11bb0-0b34-43fd-8a21-79cd356f8c28 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:39.637892 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ba951-efc3-4035-8b28-b177f3d4fd0f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:39.670894 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2353ec-6ee1-4b04-8a4c-435e1717e836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:39.671642 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2353ec-6ee1-4b04-8a4c-435e1717e836 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:39.671931 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2353ec-6ee1-4b04-8a4c-435e1717e836 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:39.672207 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2353ec-6ee1-4b04-8a4c-435e1717e836 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:39.672925 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2353ec-6ee1-4b04-8a4c-435e1717e836 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:39.673396 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2353ec-6ee1-4b04-8a4c-435e1717e836 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:39.674826 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2353ec-6ee1-4b04-8a4c-435e1717e836 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:39.686218 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 380/742] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:39.686989 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2353ec-6ee1-4b04-8a4c-435e1717e836 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:39.772305 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 363/743] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jun 30 10:14:39 2022] POST /identity/v3/services => generated 251 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:14:39.773578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f4de1-1570-47e9-b48b-d535bffe04cb None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:39.773867 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f4de1-1570-47e9-b48b-d535bffe04cb None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:39.774154 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5f4de1-1570-47e9-b48b-d535bffe04cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:39.783674 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2353ec-6ee1-4b04-8a4c-435e1717e836 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:39.804219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-bd5f4de1-1570-47e9-b48b-d535bffe04cb None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:39.805738 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-bd5f4de1-1570-47e9-b48b-d535bffe04cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:39.807645 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:39.832743 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd89138-2f3a-4559-94b6-e27bb467f9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:39.833394 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd89138-2f3a-4559-94b6-e27bb467f9e4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:39.833713 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd89138-2f3a-4559-94b6-e27bb467f9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:39.833989 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd89138-2f3a-4559-94b6-e27bb467f9e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:39.834838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd89138-2f3a-4559-94b6-e27bb467f9e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:39.835173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd89138-2f3a-4559-94b6-e27bb467f9e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:39.836730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd89138-2f3a-4559-94b6-e27bb467f9e4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:39.854220 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 381/744] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:39.854220 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd89138-2f3a-4559-94b6-e27bb467f9e4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:39.895293 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 364/745] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:39 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 30 10:14:39.895293 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f4de1-1570-47e9-b48b-d535bffe04cb None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:39.981068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd89138-2f3a-4559-94b6-e27bb467f9e4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:39.992485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:40.007886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5f4de1-1570-47e9-b48b-d535bffe04cb None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:40.023207 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e4bb6-2d3a-4268-a643-9839183cf2b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:40.024029 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e4bb6-2d3a-4268-a643-9839183cf2b8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:40.024315 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e4bb6-2d3a-4268-a643-9839183cf2b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:40.024592 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276e4bb6-2d3a-4268-a643-9839183cf2b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:40.036194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e4bb6-2d3a-4268-a643-9839183cf2b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:40.038106 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276e4bb6-2d3a-4268-a643-9839183cf2b8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:40.043119 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556aaaa-9c44-41cd-a07e-8c53165384ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:40.043973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556aaaa-9c44-41cd-a07e-8c53165384ef None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:40.044262 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556aaaa-9c44-41cd-a07e-8c53165384ef None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:40.044541 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1556aaaa-9c44-41cd-a07e-8c53165384ef None admin] PATH_INFO: `/v3/users/nova` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:40.052875 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1556aaaa-9c44-41cd-a07e-8c53165384ef None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:40.054323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1556aaaa-9c44-41cd-a07e-8c53165384ef None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:40.057264 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-276e4bb6-2d3a-4268-a643-9839183cf2b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:40.057264 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:40.057264 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:40.057264 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:40.057264 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:40.057264 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:40.057264 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:40.057264 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:40.057264 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:40.057264 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:40.057264 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:40.057264 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:40.057264 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:40.057264 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:40.057264 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:40.058154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:40.058154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:40.058154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:40.058154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:40.058154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:40.058154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:40.058154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:40.058154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:40.058154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:40.058154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:40.058154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:40.058154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:40.058154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:40.058154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:40.058154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:40.058154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:40.058154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:40.064561 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:40.064561 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:40.064561 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:40.064561 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:40.064561 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 382/746] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:40.064561 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e4bb6-2d3a-4268-a643-9839183cf2b8 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:40.070222 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-1556aaaa-9c44-41cd-a07e-8c53165384ef None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 10:14:40.070222 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:40.070222 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:40.070222 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:40.070222 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:40.070222 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:40.070222 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:40.070222 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:40.070222 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:40.070222 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:40.070222 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:40.070222 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:40.070222 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:14:40.070222 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:14:40.070222 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:14:40.070222 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:14:40.071498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:40.071498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:40.071498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:14:40.071498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:40.071498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:14:40.071498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:40.071498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:40.071498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:40.071498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:40.071498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:40.071498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:40.071498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:40.071498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:40.071498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:40.071498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:40.071498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:40.071498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:40.071498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:40.072545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:40.072545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:40.072545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:14:40.072545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:14:40.072545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:14:40.072545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:14:40.072545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:14:40.072545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:14:40.072545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 10:14:40.072545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:40.080436 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 365/747] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jun 30 10:14:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:40.081258 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556aaaa-9c44-41cd-a07e-8c53165384ef None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:40.099782 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:40.178474 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276e4bb6-2d3a-4268-a643-9839183cf2b8 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:40.188445 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1556aaaa-9c44-41cd-a07e-8c53165384ef None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:40.215435 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5446b97b-d70b-4ef7-ab3c-8c5921a655ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:40.216266 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446b97b-d70b-4ef7-ab3c-8c5921a655ed None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:40.216554 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446b97b-d70b-4ef7-ab3c-8c5921a655ed None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:40.216829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5446b97b-d70b-4ef7-ab3c-8c5921a655ed None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:40.224385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5446b97b-d70b-4ef7-ab3c-8c5921a655ed None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:40.225832 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5446b97b-d70b-4ef7-ab3c-8c5921a655ed None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:40.230481 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07285d-35af-43db-bb23-8146bde658ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:40.231391 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07285d-35af-43db-bb23-8146bde658ce None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:40.231840 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07285d-35af-43db-bb23-8146bde658ce None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:40.232415 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07285d-35af-43db-bb23-8146bde658ce None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:40.233387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07285d-35af-43db-bb23-8146bde658ce None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:40.234887 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07285d-35af-43db-bb23-8146bde658ce None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:40.241648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-5446b97b-d70b-4ef7-ab3c-8c5921a655ed None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 10:14:40.241648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:40.241648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:40.241648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:40.241648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:40.241648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:40.241648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:40.241648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:40.241648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:40.241648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:40.241648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:40.241648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:40.241648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:14:40.241648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:14:40.241648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:14:40.241648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:14:40.242787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:40.242787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:40.242787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:14:40.242787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:40.242787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:14:40.242787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:40.242787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:40.242787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:40.242787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:40.242787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:40.242787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:40.242787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:40.242787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:40.242787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:40.242787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:40.242787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:40.242787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:40.242787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:40.243859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:40.243859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:40.243859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:14:40.243859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:14:40.243859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:14:40.243859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:14:40.243859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:14:40.243859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:14:40.243859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 10:14:40.243859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:40.243859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 383/748] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jun 30 10:14:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:40.244455 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5446b97b-d70b-4ef7-ab3c-8c5921a655ed None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:40.342356 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:40.349759 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 366/749] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jun 30 10:14:40 2022] POST /identity/v3/services => generated 240 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:14:40.350677 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07285d-35af-43db-bb23-8146bde658ce None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:40.366807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:40.368634 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5446b97b-d70b-4ef7-ab3c-8c5921a655ed None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:40.414531 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece0c88-7e17-4077-9176-aecbd388589d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:40.414876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece0c88-7e17-4077-9176-aecbd388589d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:40.415162 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece0c88-7e17-4077-9176-aecbd388589d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:40.417640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece0c88-7e17-4077-9176-aecbd388589d None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:40.417640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece0c88-7e17-4077-9176-aecbd388589d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:40.417640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece0c88-7e17-4077-9176-aecbd388589d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:40.418079 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ece0c88-7e17-4077-9176-aecbd388589d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:40.435255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 384/750] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:40.435255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece0c88-7e17-4077-9176-aecbd388589d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:40.523575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07285d-35af-43db-bb23-8146bde658ce None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:40.565913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece0c88-7e17-4077-9176-aecbd388589d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:40.566598 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3df83-e121-47ea-9552-3c92ce7c65bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:40.567617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3df83-e121-47ea-9552-3c92ce7c65bd None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:40.568169 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3df83-e121-47ea-9552-3c92ce7c65bd None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:40.568452 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a3df83-e121-47ea-9552-3c92ce7c65bd None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:40.569309 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3df83-e121-47ea-9552-3c92ce7c65bd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:40.569622 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3df83-e121-47ea-9552-3c92ce7c65bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:40.571083 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a3df83-e121-47ea-9552-3c92ce7c65bd None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:40.591864 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 367/751] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jun 30 10:14:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:40.593139 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3df83-e121-47ea-9552-3c92ce7c65bd None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:40.627059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703f72f-ee2c-4893-9040-63de80b31a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:40.627394 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703f72f-ee2c-4893-9040-63de80b31a3e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:40.631287 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703f72f-ee2c-4893-9040-63de80b31a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:40.631287 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703f72f-ee2c-4893-9040-63de80b31a3e None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:40.631287 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b703f72f-ee2c-4893-9040-63de80b31a3e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:40.631287 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b703f72f-ee2c-4893-9040-63de80b31a3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:40.631287 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b703f72f-ee2c-4893-9040-63de80b31a3e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:40.650810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 385/752] 10.222.0.47 () {56 vars in 1149 bytes} [Thu Jun 30 10:14:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:40.651694 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703f72f-ee2c-4893-9040-63de80b31a3e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:40.659538 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:40.675835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:40.715134 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a3df83-e121-47ea-9552-3c92ce7c65bd None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:40.753578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fd97e-6e81-41db-8c29-b75e02c5adea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:40.754392 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fd97e-6e81-41db-8c29-b75e02c5adea None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:40.754681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fd97e-6e81-41db-8c29-b75e02c5adea None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:40.754962 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647fd97e-6e81-41db-8c29-b75e02c5adea None admin] PATH_INFO: `/v3/projects/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:40.762153 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647fd97e-6e81-41db-8c29-b75e02c5adea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:40.763734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647fd97e-6e81-41db-8c29-b75e02c5adea None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:40.764856 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703f72f-ee2c-4893-9040-63de80b31a3e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:40.778818 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-647fd97e-6e81-41db-8c29-b75e02c5adea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:40.778818 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:40.778818 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:40.778818 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:40.778818 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:40.778818 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:40.778818 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:40.778818 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:40.778818 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:40.778818 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:40.778818 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:40.778818 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:40.778818 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:14:40.778818 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:14:40.778818 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:14:40.779756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:14:40.779756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:40.779756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:40.779756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:40.779756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:40.779756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:40.779756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:40.779756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:40.779756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:40.779756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:40.779756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:40.779756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:40.779756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:40.779756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:40.779756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:40.779756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:40.779756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:40.782406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:14:40.782406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:14:40.782406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:14:40.782406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:14:40.782406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:14:40.782406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:14:40.782406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:40.782406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:40.782406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 368/753] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:40.782406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fd97e-6e81-41db-8c29-b75e02c5adea None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:40.806944 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956b329-0719-4da6-a086-f46a49dde82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:40.807771 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956b329-0719-4da6-a086-f46a49dde82e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:40.808057 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956b329-0719-4da6-a086-f46a49dde82e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:40.808334 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5956b329-0719-4da6-a086-f46a49dde82e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:40.820545 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956b329-0719-4da6-a086-f46a49dde82e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:40.822101 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5956b329-0719-4da6-a086-f46a49dde82e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:40.840294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-5956b329-0719-4da6-a086-f46a49dde82e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:40.840294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:40.840294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:40.840294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:40.840294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:40.840294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:40.840294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:40.840294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:40.840294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:40.840294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:40.840294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:40.840294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:40.840294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:40.840294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:40.840294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:40.842343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:40.842343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:40.842343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:40.842343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:40.842343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:40.842343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:40.842343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:40.842343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:40.842343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:40.842343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:40.842343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:40.842343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:40.842343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:40.842343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:40.842343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:40.842343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:40.842343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:40.844569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:40.844569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:40.844569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:40.844569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:40.847296 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 386/754] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:40.847296 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956b329-0719-4da6-a086-f46a49dde82e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:40.889844 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647fd97e-6e81-41db-8c29-b75e02c5adea None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:40.914906 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:40.918001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:40.926778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad361265-c8b3-4a8b-9ca3-d6147afc605c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:40.928203 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad361265-c8b3-4a8b-9ca3-d6147afc605c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:40.928203 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad361265-c8b3-4a8b-9ca3-d6147afc605c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:40.928485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad361265-c8b3-4a8b-9ca3-d6147afc605c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:40.973590 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5956b329-0719-4da6-a086-f46a49dde82e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:41.010520 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b834863-f01e-4ca3-8338-0e75bd12ea1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:41.011606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b834863-f01e-4ca3-8338-0e75bd12ea1d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:41.012094 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b834863-f01e-4ca3-8338-0e75bd12ea1d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:41.012567 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b834863-f01e-4ca3-8338-0e75bd12ea1d None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:41.013591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b834863-f01e-4ca3-8338-0e75bd12ea1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:41.014101 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b834863-f01e-4ca3-8338-0e75bd12ea1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:41.015772 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b834863-f01e-4ca3-8338-0e75bd12ea1d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:41.025982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad361265-c8b3-4a8b-9ca3-d6147afc605c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:41.028264 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad361265-c8b3-4a8b-9ca3-d6147afc605c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:41.030678 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 387/755] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:14:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:41.031925 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b834863-f01e-4ca3-8338-0e75bd12ea1d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:41.149108 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b834863-f01e-4ca3-8338-0e75bd12ea1d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:41.155703 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 369/756] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:14:40 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 10:14:41.156782 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad361265-c8b3-4a8b-9ca3-d6147afc605c None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:41.170566 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:41.192994 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e243c-dbcd-464a-a31d-f89bd01e7b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:41.193665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:41.194196 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e243c-dbcd-464a-a31d-f89bd01e7b73 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:41.194703 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e243c-dbcd-464a-a31d-f89bd01e7b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:41.195198 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e243c-dbcd-464a-a31d-f89bd01e7b73 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:41.196170 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e243c-dbcd-464a-a31d-f89bd01e7b73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:41.196677 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e243c-dbcd-464a-a31d-f89bd01e7b73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:41.198307 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e243c-dbcd-464a-a31d-f89bd01e7b73 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:41.210949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 388/757] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:41.212261 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e243c-dbcd-464a-a31d-f89bd01e7b73 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:41.293686 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad361265-c8b3-4a8b-9ca3-d6147afc605c None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:41.317997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e243c-dbcd-464a-a31d-f89bd01e7b73 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:41.345283 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824a919-0a16-4f25-b718-33922fd0eb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:41.346044 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824a919-0a16-4f25-b718-33922fd0eb0e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:41.346334 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824a919-0a16-4f25-b718-33922fd0eb0e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:41.346636 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824a919-0a16-4f25-b718-33922fd0eb0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:41.348251 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824a919-0a16-4f25-b718-33922fd0eb0e None admin] RBAC: Adding query filter params (role.id=e63bda042e104a4a93fba15ffcfe1183, scope.project.id=50902e0539434c82b505dc5009aeaf85, user.id=07de926cb8594b098f79111dab8a9cc6) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:41.348582 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824a919-0a16-4f25-b718-33922fd0eb0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:41.350378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824a919-0a16-4f25-b718-33922fd0eb0e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:41.358864 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 370/758] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:14:41 2022] GET /identity/v3/role_assignments?user.id=07de926cb8594b098f79111dab8a9cc6&scope.project.id=50902e0539434c82b505dc5009aeaf85&role.id=e63bda042e104a4a93fba15ffcfe1183 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:41.359730 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824a919-0a16-4f25-b718-33922fd0eb0e None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:41.360322 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23067eb-9ad2-4411-857e-9f30c2518d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:41.361615 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23067eb-9ad2-4411-857e-9f30c2518d3b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:41.362120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23067eb-9ad2-4411-857e-9f30c2518d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:41.362621 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23067eb-9ad2-4411-857e-9f30c2518d3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:41.375791 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23067eb-9ad2-4411-857e-9f30c2518d3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:41.378862 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23067eb-9ad2-4411-857e-9f30c2518d3b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:41.398509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-a23067eb-9ad2-4411-857e-9f30c2518d3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:41.398509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:41.398509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:41.398509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:41.398509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:41.398509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:41.398509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:41.398509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:41.398509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:41.398509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:41.398509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:41.398509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:41.398509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:41.398509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:41.398509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:41.399818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:41.399818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:41.399818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:41.399818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:41.399818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:41.399818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:41.399818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:41.399818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:41.399818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:41.399818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:41.399818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:41.399818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:41.399818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:41.399818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:41.399818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:41.399818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:41.399818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:41.400717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:41.400717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:41.400717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:41.400717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:41.402951 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 389/759] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:41.414517 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23067eb-9ad2-4411-857e-9f30c2518d3b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:41.479789 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:41.480020 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:41.501226 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824a919-0a16-4f25-b718-33922fd0eb0e None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:41.552244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23067eb-9ad2-4411-857e-9f30c2518d3b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:41.581420 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17437a9-1d9c-422f-964e-477d8f413740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:41.582256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17437a9-1d9c-422f-964e-477d8f413740 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:41.582541 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17437a9-1d9c-422f-964e-477d8f413740 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:41.582827 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17437a9-1d9c-422f-964e-477d8f413740 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:41.597829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17437a9-1d9c-422f-964e-477d8f413740 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:41.599338 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17437a9-1d9c-422f-964e-477d8f413740 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:41.616343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-f17437a9-1d9c-422f-964e-477d8f413740 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:41.616343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:41.616343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:41.616343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:41.616343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:41.616343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:41.616343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:41.616343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:41.616343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:41.616343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:41.616343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:41.616343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:41.616343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:14:41.616343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:14:41.616343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:14:41.617447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:14:41.617447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:41.617447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:41.617447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:41.617447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:41.617447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:41.617447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:41.617447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:41.617447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:41.617447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:41.617447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:41.617447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:41.617447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:41.617447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:41.617447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:41.617447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:41.617447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:41.618334 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:14:41.618334 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:14:41.618334 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:14:41.618334 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:14:41.618334 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:14:41.618334 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:14:41.618334 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:41.618334 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:41.618334 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 371/760] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:41.624730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8de8a-d869-4771-8e51-6dde06c9abaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:41.624915 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17437a9-1d9c-422f-964e-477d8f413740 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:41.625514 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8de8a-d869-4771-8e51-6dde06c9abaf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:41.625805 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8de8a-d869-4771-8e51-6dde06c9abaf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:41.626083 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e8de8a-d869-4771-8e51-6dde06c9abaf None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:41.627450 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8de8a-d869-4771-8e51-6dde06c9abaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:41.627764 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8de8a-d869-4771-8e51-6dde06c9abaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:41.629415 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e8de8a-d869-4771-8e51-6dde06c9abaf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:41.648633 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 390/761] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:41.656513 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8de8a-d869-4771-8e51-6dde06c9abaf None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:41.821929 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:41.829066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17437a9-1d9c-422f-964e-477d8f413740 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:41.849940 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e8de8a-d869-4771-8e51-6dde06c9abaf None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:41.878218 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d6548-76cb-4d88-b902-6c0fa59700ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:41.878889 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d6548-76cb-4d88-b902-6c0fa59700ba None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:41.879172 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d6548-76cb-4d88-b902-6c0fa59700ba None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:41.879606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1d6548-76cb-4d88-b902-6c0fa59700ba None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:41.880412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d6548-76cb-4d88-b902-6c0fa59700ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:41.880725 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d6548-76cb-4d88-b902-6c0fa59700ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:41.882434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1d6548-76cb-4d88-b902-6c0fa59700ba None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:41.896578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 372/762] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:14:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:41.902766 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d6548-76cb-4d88-b902-6c0fa59700ba None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:41.920446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:41.979413 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945baf33-3db7-4193-adc5-23780ceefacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:41.980270 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945baf33-3db7-4193-adc5-23780ceefacc None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:41.980558 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945baf33-3db7-4193-adc5-23780ceefacc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:41.980834 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945baf33-3db7-4193-adc5-23780ceefacc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:41.995577 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945baf33-3db7-4193-adc5-23780ceefacc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:41.999031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945baf33-3db7-4193-adc5-23780ceefacc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:42.014405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-945baf33-3db7-4193-adc5-23780ceefacc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:42.014405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:42.014405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:42.014405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:42.014405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:42.014405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:42.014405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:42.014405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:42.014405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:42.014405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:42.014405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:42.014405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:42.014405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:14:42.014405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:14:42.014405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:14:42.015419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:14:42.015419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:42.015419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:42.015419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:42.015419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:42.015419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:42.015419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:42.015419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:42.015419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:42.015419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:42.015419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:42.015419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:42.015419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:42.015419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:42.015419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:42.015419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:42.015419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:42.016341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:14:42.016341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:14:42.016341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:14:42.016341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:14:42.016341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:14:42.016341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:14:42.016341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:42.016341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:42.016341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 391/763] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:42.018111 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960a0a6-7afe-4162-9322-970ce028a735 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:42.018407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960a0a6-7afe-4162-9322-970ce028a735 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:42.023496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d960a0a6-7afe-4162-9322-970ce028a735 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:42.023496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 392/764] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:42.032826 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee44c2-5b62-45e6-b5e0-45f7824710c1 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:42.033122 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee44c2-5b62-45e6-b5e0-45f7824710c1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:42.033376 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eee44c2-5b62-45e6-b5e0-45f7824710c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:42.041054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1d6548-76cb-4d88-b902-6c0fa59700ba None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:42.057738 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7eee44c2-5b62-45e6-b5e0-45f7824710c1 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:42.059290 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7eee44c2-5b62-45e6-b5e0-45f7824710c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:42.085857 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57dd3d-a427-412c-b579-d6ce5fac6c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:42.086763 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57dd3d-a427-412c-b579-d6ce5fac6c13 None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:42.087152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57dd3d-a427-412c-b579-d6ce5fac6c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:42.087469 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57dd3d-a427-412c-b579-d6ce5fac6c13 None admin] PATH_INFO: `/v3/projects/50902e0539434c82b505dc5009aeaf85/users/71f6d7715e6f49ffb9fdf6e1307bbd51/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:42.089681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57dd3d-a427-412c-b579-d6ce5fac6c13 None admin] RBAC: Authorizing `identity:create_grant(project_id=50902e0539434c82b505dc5009aeaf85, user_id=71f6d7715e6f49ffb9fdf6e1307bbd51, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:42.092155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57dd3d-a427-412c-b579-d6ce5fac6c13 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:42.133106 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 393/765] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:14:42.136148 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee44c2-5b62-45e6-b5e0-45f7824710c1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:42.195723 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 373/766] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jun 30 10:14:41 2022] PUT /identity/v3/projects/50902e0539434c82b505dc5009aeaf85/users/71f6d7715e6f49ffb9fdf6e1307bbd51/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:14:42.196497 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57dd3d-a427-412c-b579-d6ce5fac6c13 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:42.234080 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:42.266026 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eee44c2-5b62-45e6-b5e0-45f7824710c1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:42.313048 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b1513-af09-4136-b78f-4e18f7c861a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:42.314612 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b1513-af09-4136-b78f-4e18f7c861a3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:42.314942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b1513-af09-4136-b78f-4e18f7c861a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:42.315368 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b1513-af09-4136-b78f-4e18f7c861a3 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:42.316529 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b1513-af09-4136-b78f-4e18f7c861a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:42.316854 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b1513-af09-4136-b78f-4e18f7c861a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:42.318409 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b1513-af09-4136-b78f-4e18f7c861a3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:42.334609 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 394/767] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:14:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:42.344578 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b1513-af09-4136-b78f-4e18f7c861a3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:42.383071 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:42.441748 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57dd3d-a427-412c-b579-d6ce5fac6c13 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:42.490654 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccec5a0-e9b5-4e67-8ebd-9d323620df3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:42.491179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccec5a0-e9b5-4e67-8ebd-9d323620df3b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:42.491657 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccec5a0-e9b5-4e67-8ebd-9d323620df3b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:42.491943 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccec5a0-e9b5-4e67-8ebd-9d323620df3b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:42.492917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccec5a0-e9b5-4e67-8ebd-9d323620df3b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:42.499287 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccec5a0-e9b5-4e67-8ebd-9d323620df3b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:42.515904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-dccec5a0-e9b5-4e67-8ebd-9d323620df3b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 10:14:42.515904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:42.515904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 10:14:42.515904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 10:14:42.515904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 10:14:42.515904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 10:14:42.515904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 10:14:42.515904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:14:42.515904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 10:14:42.515904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:42.515904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:14:42.515904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:42.515904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:42.515904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:42.515904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:42.515904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:42.517485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:42.517485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:42.517485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:42.517485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:42.517485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:42.517485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:42.517485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:42.517485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 10:14:42.517485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 10:14:42.517485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 10:14:42.517485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 10:14:42.517485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:42.517485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:42.517485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:42.517485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:42.517485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:42.518465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:42.518465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:42.518465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:42.518465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:42.518465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:42.518465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:42.518465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:42.518465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:42.518465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:42.518465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:42.518465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:42.518465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 10:14:42.518465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:14:42.518465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 10:14:42.518465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:42.518465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 374/768] 10.222.0.47 () {56 vars in 1129 bytes} [Thu Jun 30 10:14:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:42.521169 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccec5a0-e9b5-4e67-8ebd-9d323620df3b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:42.543403 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b1513-af09-4136-b78f-4e18f7c861a3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:42.590144 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee0a345-1f80-4e90-a4c3-0ad1d10d9c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:42.590997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0a345-1f80-4e90-a4c3-0ad1d10d9c4a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:42.591307 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0a345-1f80-4e90-a4c3-0ad1d10d9c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:42.591593 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee0a345-1f80-4e90-a4c3-0ad1d10d9c4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:42.593060 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee0a345-1f80-4e90-a4c3-0ad1d10d9c4a None admin] RBAC: Adding query filter params (role.id=99d8cd9cd87e472290bf0de2710b5857, scope.project.id=50902e0539434c82b505dc5009aeaf85, user.id=66ad8defb9b6453c96f1a8a7343695ba) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:42.593375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee0a345-1f80-4e90-a4c3-0ad1d10d9c4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:42.596670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee0a345-1f80-4e90-a4c3-0ad1d10d9c4a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:42.600446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:42.605020 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 395/769] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:14:42 2022] GET /identity/v3/role_assignments?user.id=66ad8defb9b6453c96f1a8a7343695ba&scope.project.id=50902e0539434c82b505dc5009aeaf85&role.id=99d8cd9cd87e472290bf0de2710b5857 => generated 259 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:42.607350 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c6512-09e4-434a-8716-bc31aa504cfc None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:42.607650 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c6512-09e4-434a-8716-bc31aa504cfc None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:42.607931 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c6512-09e4-434a-8716-bc31aa504cfc None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:42.628086 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 396/770] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:42.628086 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea78e4f-d5f0-4c2e-81f6-b31d52912295 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:42.628086 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea78e4f-d5f0-4c2e-81f6-b31d52912295 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:42.628086 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea78e4f-d5f0-4c2e-81f6-b31d52912295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:42.656648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5ea78e4f-d5f0-4c2e-81f6-b31d52912295 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:42.657936 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5ea78e4f-d5f0-4c2e-81f6-b31d52912295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:42.676976 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccec5a0-e9b5-4e67-8ebd-9d323620df3b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:42.735062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:42.749371 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6174c5-e56d-4f0c-b005-d82bde51b95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:42.749371 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6174c5-e56d-4f0c-b005-d82bde51b95d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:42.749371 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6174c5-e56d-4f0c-b005-d82bde51b95d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:42.749371 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6174c5-e56d-4f0c-b005-d82bde51b95d None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:42.749787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6174c5-e56d-4f0c-b005-d82bde51b95d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:42.750004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6174c5-e56d-4f0c-b005-d82bde51b95d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:42.752392 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6174c5-e56d-4f0c-b005-d82bde51b95d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:42.758717 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 375/771] 10.222.0.47 () {56 vars in 1124 bytes} [Thu Jun 30 10:14:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:42.765150 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6174c5-e56d-4f0c-b005-d82bde51b95d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:42.788294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 397/772] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:14:42.811398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea78e4f-d5f0-4c2e-81f6-b31d52912295 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:42.876745 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6174c5-e56d-4f0c-b005-d82bde51b95d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:42.919720 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea78e4f-d5f0-4c2e-81f6-b31d52912295 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:42.932894 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbe7b9-ca27-4e5b-8529-2ebdf30c135e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:42.933579 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbe7b9-ca27-4e5b-8529-2ebdf30c135e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:42.933865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbe7b9-ca27-4e5b-8529-2ebdf30c135e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:42.934145 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fbe7b9-ca27-4e5b-8529-2ebdf30c135e None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:42.934867 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbe7b9-ca27-4e5b-8529-2ebdf30c135e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:42.935180 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbe7b9-ca27-4e5b-8529-2ebdf30c135e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:42.936655 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fbe7b9-ca27-4e5b-8529-2ebdf30c135e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:42.949646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 376/773] 10.222.0.47 () {56 vars in 1124 bytes} [Thu Jun 30 10:14:42 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:42.950335 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:42.959556 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbe7b9-ca27-4e5b-8529-2ebdf30c135e None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:42.972543 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda34a4e-7986-4969-932a-16b13cb5548c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:42.972543 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda34a4e-7986-4969-932a-16b13cb5548c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:42.972543 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda34a4e-7986-4969-932a-16b13cb5548c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:42.972543 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda34a4e-7986-4969-932a-16b13cb5548c None admin] PATH_INFO: `/v3/services/network` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:42.972882 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda34a4e-7986-4969-932a-16b13cb5548c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:42.972882 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda34a4e-7986-4969-932a-16b13cb5548c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:42.994905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-cda34a4e-7986-4969-932a-16b13cb5548c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 10:14:42.994905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:42.994905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 10:14:42.994905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 10:14:42.994905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 10:14:42.994905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 10:14:42.994905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 10:14:42.994905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:14:42.994905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 10:14:42.994905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:42.994905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:14:42.994905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:42.994905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:42.994905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:42.994905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:42.994905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:42.994905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:42.995988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:42.995988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:42.995988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:42.995988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:42.995988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:42.995988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:42.995988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 10:14:42.995988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 10:14:42.995988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 10:14:42.995988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 10:14:42.995988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:42.995988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:42.995988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:42.995988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:42.995988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:42.995988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:42.996838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:42.996838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:42.996838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:42.996838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:42.996838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:42.996838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:42.996838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:42.996838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:42.996838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:42.996838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:42.996838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 10:14:42.996838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:14:42.996838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 10:14:42.996838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:43.005074 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 398/774] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:42 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:14:43.012214 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda34a4e-7986-4969-932a-16b13cb5548c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:43.038700 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:43.075487 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fbe7b9-ca27-4e5b-8529-2ebdf30c135e None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:43.111235 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb9a6d-6477-4c99-94fd-ad44fe55ab11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:43.111942 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb9a6d-6477-4c99-94fd-ad44fe55ab11 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:43.112232 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb9a6d-6477-4c99-94fd-ad44fe55ab11 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:43.112578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bb9a6d-6477-4c99-94fd-ad44fe55ab11 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:43.115043 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb9a6d-6477-4c99-94fd-ad44fe55ab11 None admin] RBAC: Adding query filter params (interface=public, service_id=0cd8486bde6b495e80b85405e802b917, region_id=RegionOne) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:43.115488 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb9a6d-6477-4c99-94fd-ad44fe55ab11 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:43.116879 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bb9a6d-6477-4c99-94fd-ad44fe55ab11 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:43.127226 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 377/775] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Jun 30 10:14:42 2022] GET /identity/v3/endpoints?service_id=0cd8486bde6b495e80b85405e802b917&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:14:43.135747 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb9a6d-6477-4c99-94fd-ad44fe55ab11 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:43.194388 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda34a4e-7986-4969-932a-16b13cb5548c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:43.234127 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c2ef3-c999-40fa-8f24-831aa4b58d66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:43.234796 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c2ef3-c999-40fa-8f24-831aa4b58d66 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:43.235081 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c2ef3-c999-40fa-8f24-831aa4b58d66 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:43.235387 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513c2ef3-c999-40fa-8f24-831aa4b58d66 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:43.236149 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513c2ef3-c999-40fa-8f24-831aa4b58d66 None admin] RBAC: Adding query filter params (name=network) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:43.236496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513c2ef3-c999-40fa-8f24-831aa4b58d66 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:43.239006 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513c2ef3-c999-40fa-8f24-831aa4b58d66 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:43.247139 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 399/776] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jun 30 10:14:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:43.252985 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c2ef3-c999-40fa-8f24-831aa4b58d66 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:43.316161 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:43.359570 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513c2ef3-c999-40fa-8f24-831aa4b58d66 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:43.366439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bb9a6d-6477-4c99-94fd-ad44fe55ab11 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:43.395210 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d88cb-ed77-4ea5-8441-4f590151de2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:43.395966 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d88cb-ed77-4ea5-8441-4f590151de2f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:43.396253 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d88cb-ed77-4ea5-8441-4f590151de2f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:43.396534 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1d88cb-ed77-4ea5-8441-4f590151de2f None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:43.397545 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d88cb-ed77-4ea5-8441-4f590151de2f None admin] RBAC: Adding query filter params (type=network) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:43.397861 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d88cb-ed77-4ea5-8441-4f590151de2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:43.399241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1d88cb-ed77-4ea5-8441-4f590151de2f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:43.404686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 400/777] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jun 30 10:14:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:43.410607 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d88cb-ed77-4ea5-8441-4f590151de2f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:43.427629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:43.450621 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d8727-8493-46ad-97a4-6a54833e2ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:43.451376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d8727-8493-46ad-97a4-6a54833e2ed2 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:43.451666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d8727-8493-46ad-97a4-6a54833e2ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:43.451946 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d8727-8493-46ad-97a4-6a54833e2ed2 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:43.452634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002d8727-8493-46ad-97a4-6a54833e2ed2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:43.456309 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002d8727-8493-46ad-97a4-6a54833e2ed2 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:43.465934 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 378/778] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jun 30 10:14:43 2022] GET /identity/v3/services => generated 772 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:43.617821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1d88cb-ed77-4ea5-8441-4f590151de2f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:43.651636 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:43.667652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42d83f8-d65e-43dc-92da-e0330c51c4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:43.668573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d83f8-d65e-43dc-92da-e0330c51c4a6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:43.669018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d83f8-d65e-43dc-92da-e0330c51c4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:43.669445 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d83f8-d65e-43dc-92da-e0330c51c4a6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:43.670399 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42d83f8-d65e-43dc-92da-e0330c51c4a6 None admin] RBAC: Adding query filter params (interface=public, service_id=d844e67dda3b48e6956c845cee8f474e, region_id=RegionOne) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:43.670872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42d83f8-d65e-43dc-92da-e0330c51c4a6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:43.672648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42d83f8-d65e-43dc-92da-e0330c51c4a6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:43.683002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 401/779] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Jun 30 10:14:43 2022] GET /identity/v3/endpoints?service_id=d844e67dda3b48e6956c845cee8f474e&interface=public®ion_id=RegionOne => generated 194 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:43.686112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d8727-8493-46ad-97a4-6a54833e2ed2 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:43.795637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d8727-8493-46ad-97a4-6a54833e2ed2 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:43.839433 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863d1b34-bfd7-459e-9559-a6b508bbf1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:43.840126 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d1b34-bfd7-459e-9559-a6b508bbf1c0 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:43.840420 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d1b34-bfd7-459e-9559-a6b508bbf1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:43.840708 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863d1b34-bfd7-459e-9559-a6b508bbf1c0 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:43.841403 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d1b34-bfd7-459e-9559-a6b508bbf1c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:43.842782 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863d1b34-bfd7-459e-9559-a6b508bbf1c0 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:43.848955 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 379/780] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jun 30 10:14:43 2022] GET /identity/v3/services => generated 772 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:43.886602 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba73db-fb75-4879-bbe7-35fadbc73281 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:43.886887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba73db-fb75-4879-bbe7-35fadbc73281 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:43.887091 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba73db-fb75-4879-bbe7-35fadbc73281 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:43.888552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 402/781] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:43.902768 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610dca0-a64e-4777-85dd-1345783e1ba3 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:43.903091 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610dca0-a64e-4777-85dd-1345783e1ba3 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:43.903365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2610dca0-a64e-4777-85dd-1345783e1ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:43.930443 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-2610dca0-a64e-4777-85dd-1345783e1ba3 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:43.931676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-2610dca0-a64e-4777-85dd-1345783e1ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:44.003535 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:44.034691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 380/782] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:14:44.069808 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba73db-fb75-4879-bbe7-35fadbc73281 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:44.186281 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba73db-fb75-4879-bbe7-35fadbc73281 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:44.224544 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333db6e8-06f5-46d9-b49d-4e7466fbd384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:44.225720 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333db6e8-06f5-46d9-b49d-4e7466fbd384 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:44.226266 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333db6e8-06f5-46d9-b49d-4e7466fbd384 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:44.226735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333db6e8-06f5-46d9-b49d-4e7466fbd384 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:44.239997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333db6e8-06f5-46d9-b49d-4e7466fbd384 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:44.248071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-333db6e8-06f5-46d9-b49d-4e7466fbd384 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:14:44.248071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:44.248071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:44.248071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:44.248071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:44.248071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:44.248071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:44.248071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:44.248071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:44.248071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:44.248071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:44.248071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:44.248071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:44.248071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:44.248071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:44.248071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:44.248978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:44.248978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:44.248978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:44.248978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:44.248978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:44.248978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:44.248978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:44.248978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:44.248978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:44.248978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:44.248978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:44.248978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:44.248978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:44.248978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:44.248978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:44.248978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:44.249826 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:44.249826 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:44.249826 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:44.249826 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:44.249826 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:44.249826 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:44.249826 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:44.249826 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:44.249826 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:44.249826 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:44.249826 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:14:44.249826 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:44.253234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 403/783] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:14:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:44.258873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2610dca0-a64e-4777-85dd-1345783e1ba3 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:44.414056 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2610dca0-a64e-4777-85dd-1345783e1ba3 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:44.450347 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720684fb-88dc-4e87-975e-ae4ea96ff3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:44.451454 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720684fb-88dc-4e87-975e-ae4ea96ff3be None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:44.451956 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720684fb-88dc-4e87-975e-ae4ea96ff3be None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:44.452425 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720684fb-88dc-4e87-975e-ae4ea96ff3be None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:44.453366 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720684fb-88dc-4e87-975e-ae4ea96ff3be None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:44.453891 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720684fb-88dc-4e87-975e-ae4ea96ff3be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:44.456092 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720684fb-88dc-4e87-975e-ae4ea96ff3be None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:44.465430 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 381/784] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:14:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:44.471472 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333db6e8-06f5-46d9-b49d-4e7466fbd384 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:44.658190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:44.722475 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333db6e8-06f5-46d9-b49d-4e7466fbd384 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:44.761351 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86537a-fd0e-411b-a68e-f93eba67cbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:44.762145 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86537a-fd0e-411b-a68e-f93eba67cbdd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:44.762423 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86537a-fd0e-411b-a68e-f93eba67cbdd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:44.762692 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e86537a-fd0e-411b-a68e-f93eba67cbdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:44.776729 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e86537a-fd0e-411b-a68e-f93eba67cbdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:44.778182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e86537a-fd0e-411b-a68e-f93eba67cbdd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:44.801727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-3e86537a-fd0e-411b-a68e-f93eba67cbdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:44.801727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:44.801727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:44.801727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:44.801727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:44.801727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:44.801727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:44.801727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:44.801727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:44.801727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:44.801727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:44.801727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:44.801727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:44.801727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:44.801727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:44.802647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:44.802647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:44.802647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:44.802647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:44.802647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:44.802647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:44.802647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:44.802647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:44.802647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:44.802647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:44.802647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:44.802647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:44.802647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:44.802647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:44.802647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:44.802647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:44.802647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:44.803565 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:44.803565 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:44.803565 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:44.803565 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:44.803565 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 404/785] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:44.808730 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720684fb-88dc-4e87-975e-ae4ea96ff3be None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:44.859323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:44.915539 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720684fb-88dc-4e87-975e-ae4ea96ff3be None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:44.950483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9957c70f-ea2a-4762-9a15-744db34a6a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:44.951154 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957c70f-ea2a-4762-9a15-744db34a6a32 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:44.951461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957c70f-ea2a-4762-9a15-744db34a6a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:44.951736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9957c70f-ea2a-4762-9a15-744db34a6a32 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:44.952457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9957c70f-ea2a-4762-9a15-744db34a6a32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:44.952760 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9957c70f-ea2a-4762-9a15-744db34a6a32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:44.954192 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9957c70f-ea2a-4762-9a15-744db34a6a32 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:44.966722 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 382/786] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:44.973139 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86537a-fd0e-411b-a68e-f93eba67cbdd None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:45.033299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ee3c2-279d-4382-b00d-e6317152aee7 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:45.033299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ee3c2-279d-4382-b00d-e6317152aee7 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:45.033711 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703ee3c2-279d-4382-b00d-e6317152aee7 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:45.035050 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 383/787] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:14:45.041145 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373c412-40c3-49e8-8f7a-00aa3a119c4a None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:45.041484 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373c412-40c3-49e8-8f7a-00aa3a119c4a None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:45.041761 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373c412-40c3-49e8-8f7a-00aa3a119c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:45.064081 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-4373c412-40c3-49e8-8f7a-00aa3a119c4a None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:45.065345 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-4373c412-40c3-49e8-8f7a-00aa3a119c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:45.115377 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:45.119336 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e86537a-fd0e-411b-a68e-f93eba67cbdd None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:45.147540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 384/788] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 10:14:45.148980 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0649b-a89d-4e97-b0ce-26e94571ed2f None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:45.149276 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0649b-a89d-4e97-b0ce-26e94571ed2f None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:45.149555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0649b-a89d-4e97-b0ce-26e94571ed2f None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:45.151108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 385/789] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:45.154986 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd93b349-8b81-43e2-aa13-2ad4bf76563f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:45.155810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93b349-8b81-43e2-aa13-2ad4bf76563f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:45.156302 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93b349-8b81-43e2-aa13-2ad4bf76563f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:45.156633 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93b349-8b81-43e2-aa13-2ad4bf76563f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:45.165051 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd93b349-8b81-43e2-aa13-2ad4bf76563f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:45.166780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd93b349-8b81-43e2-aa13-2ad4bf76563f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:45.167097 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6934945-39d3-4250-8516-f058fce2309a None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:45.167494 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6934945-39d3-4250-8516-f058fce2309a None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:45.167772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6934945-39d3-4250-8516-f058fce2309a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:45.183439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-bd93b349-8b81-43e2-aa13-2ad4bf76563f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 10:14:45.183439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:45.183439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:45.183439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:45.183439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:45.183439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:45.183439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:45.183439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:45.183439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:45.183439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:45.183439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:45.183439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:45.183439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:14:45.183439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:14:45.183439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:14:45.183439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:14:45.187689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:45.187689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:45.187689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:14:45.187689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:45.187689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:14:45.187689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:45.187689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:45.187689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:45.187689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:45.187689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:45.187689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:45.187689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:45.187689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:45.187689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:45.187689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:45.187689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:45.187689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:45.187689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:45.189033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:45.189033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:45.189033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:14:45.189033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:14:45.189033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:14:45.189033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:14:45.189033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:14:45.189033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:14:45.189033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 10:14:45.189033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:45.189033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 405/790] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:14:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:45.189033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd93b349-8b81-43e2-aa13-2ad4bf76563f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:45.189738 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-d6934945-39d3-4250-8516-f058fce2309a None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:45.189738 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d6934945-39d3-4250-8516-f058fce2309a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:45.222007 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:45.273961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 386/791] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 10:14:45.275026 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6934945-39d3-4250-8516-f058fce2309a None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:45.330902 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd93b349-8b81-43e2-aa13-2ad4bf76563f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:45.374884 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ebb006-6ead-4d52-bea0-f79722dbb382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:45.376728 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ebb006-6ead-4d52-bea0-f79722dbb382 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:45.377108 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ebb006-6ead-4d52-bea0-f79722dbb382 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:45.377393 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ebb006-6ead-4d52-bea0-f79722dbb382 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:45.393070 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ebb006-6ead-4d52-bea0-f79722dbb382 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:45.402493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-07ebb006-6ead-4d52-bea0-f79722dbb382 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:14:45.402493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:45.402493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:45.402493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:45.402493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:45.402493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:45.402493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:45.402493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:45.402493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:45.402493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:45.402493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:45.402493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:45.402493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:45.402493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:45.402493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:45.402493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:45.403503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:45.403503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:45.403503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:45.403503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:45.403503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:45.403503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:45.403503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:45.403503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:45.403503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:45.403503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:45.403503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:45.403503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:45.403503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:45.403503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:45.403503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:45.403503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:45.404381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:45.404381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:45.404381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:45.404381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:45.404381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:45.404381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:45.404381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:45.404381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:45.404381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:45.404381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:45.404381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:14:45.404381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:45.404381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 406/792] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:14:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:45.405090 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ebb006-6ead-4d52-bea0-f79722dbb382 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:45.415949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6934945-39d3-4250-8516-f058fce2309a None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:45.430801 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:45.459693 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e1a85-d102-4e9a-a0da-eab094c21e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:45.460588 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e1a85-d102-4e9a-a0da-eab094c21e3f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:45.460979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e1a85-d102-4e9a-a0da-eab094c21e3f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:45.461463 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9e1a85-d102-4e9a-a0da-eab094c21e3f None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:45.462785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e1a85-d102-4e9a-a0da-eab094c21e3f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:45.463221 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e1a85-d102-4e9a-a0da-eab094c21e3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:45.465127 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9e1a85-d102-4e9a-a0da-eab094c21e3f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:45.493755 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 387/793] 10.222.0.47 () {56 vars in 1135 bytes} [Thu Jun 30 10:14:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:45.495026 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e1a85-d102-4e9a-a0da-eab094c21e3f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:45.532130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:45.535056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ebb006-6ead-4d52-bea0-f79722dbb382 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:45.570270 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8410e7b-c780-41e0-83c3-fecc78ad6b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:45.571236 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8410e7b-c780-41e0-83c3-fecc78ad6b95 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:45.571634 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8410e7b-c780-41e0-83c3-fecc78ad6b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:45.572021 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8410e7b-c780-41e0-83c3-fecc78ad6b95 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:45.583418 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8410e7b-c780-41e0-83c3-fecc78ad6b95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:45.590993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-e8410e7b-c780-41e0-83c3-fecc78ad6b95 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:45.590993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:45.590993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:45.590993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:45.590993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:45.590993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:45.590993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:45.590993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:45.590993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:45.590993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:45.590993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:45.590993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:45.590993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:45.590993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:45.590993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:45.590993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:45.592112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:45.592112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:45.592112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:45.592112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:45.592112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:45.592112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:45.592112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:45.592112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:45.592112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:45.592112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:45.592112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:45.592112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:45.592112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:45.592112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:45.592112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:45.592112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:45.592976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:45.592976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:45.592976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:45.592976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:45.592976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:45.592976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:45.592976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:45.592976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:45.592976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:45.592976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:45.592976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:45.592976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:45.598804 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 407/794] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jun 30 10:14:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:45.599820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8410e7b-c780-41e0-83c3-fecc78ad6b95 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:45.621438 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9e1a85-d102-4e9a-a0da-eab094c21e3f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:45.661108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eabf01-2459-413d-978b-88aa5e1dc729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:45.662206 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eabf01-2459-413d-978b-88aa5e1dc729 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:45.662675 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eabf01-2459-413d-978b-88aa5e1dc729 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:45.663153 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eabf01-2459-413d-978b-88aa5e1dc729 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:45.664218 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eabf01-2459-413d-978b-88aa5e1dc729 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:45.664730 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eabf01-2459-413d-978b-88aa5e1dc729 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:45.666389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eabf01-2459-413d-978b-88aa5e1dc729 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:45.676133 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 388/795] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:14:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:45.677433 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eabf01-2459-413d-978b-88aa5e1dc729 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:45.682176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:45.704781 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8410e7b-c780-41e0-83c3-fecc78ad6b95 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:45.740287 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a2734-6479-4405-9989-0648c8a50b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:45.741172 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a2734-6479-4405-9989-0648c8a50b89 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:45.741525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a2734-6479-4405-9989-0648c8a50b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:45.741870 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a2734-6479-4405-9989-0648c8a50b89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:45.752313 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a2734-6479-4405-9989-0648c8a50b89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:45.753864 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a2734-6479-4405-9989-0648c8a50b89 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:45.772327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-109a2734-6479-4405-9989-0648c8a50b89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:45.772327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:45.772327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:45.772327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:45.772327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:45.772327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:45.772327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:45.772327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:45.772327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:45.772327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:45.772327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:45.772327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:45.772327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:45.772327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:45.772327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:45.773285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:45.773285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:45.773285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:45.773285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:45.773285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:45.773285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:45.773285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:45.773285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:45.773285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:45.773285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:45.773285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:45.773285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:45.773285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:45.773285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:45.773285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:45.773285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:45.773285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:45.774176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:45.774176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:45.774176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:45.774176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:45.774176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 408/796] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:45.775441 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a2734-6479-4405-9989-0648c8a50b89 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:45.778601 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:45.798528 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eabf01-2459-413d-978b-88aa5e1dc729 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:45.835289 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4708b4-0266-4228-8960-0ae528a5cb9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:45.836800 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4708b4-0266-4228-8960-0ae528a5cb9a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:45.837271 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4708b4-0266-4228-8960-0ae528a5cb9a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:45.837729 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4708b4-0266-4228-8960-0ae528a5cb9a None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:45.838843 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4708b4-0266-4228-8960-0ae528a5cb9a None admin] RBAC: Adding query filter params (name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:45.839457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4708b4-0266-4228-8960-0ae528a5cb9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:45.841115 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4708b4-0266-4228-8960-0ae528a5cb9a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:45.851771 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 389/797] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jun 30 10:14:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:45.852844 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4708b4-0266-4228-8960-0ae528a5cb9a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:45.887677 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a2734-6479-4405-9989-0648c8a50b89 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:45.924931 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:45.932725 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184c907-868c-48e7-9454-edd27d8adf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:45.933879 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184c907-868c-48e7-9454-edd27d8adf95 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:45.934215 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184c907-868c-48e7-9454-edd27d8adf95 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:45.942093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184c907-868c-48e7-9454-edd27d8adf95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:45.948335 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6184c907-868c-48e7-9454-edd27d8adf95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:45.949918 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6184c907-868c-48e7-9454-edd27d8adf95 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:45.969587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-6184c907-868c-48e7-9454-edd27d8adf95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:45.969587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:45.969587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:45.969587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:45.969587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:45.969587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:45.969587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:45.969587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:45.969587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:45.969587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:45.969587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:45.969587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:45.969587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:45.969587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:45.969587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:45.970686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:45.970686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:45.970686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:45.970686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:45.970686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:45.970686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:45.970686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:45.970686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:45.970686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:45.970686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:45.970686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:45.970686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:45.970686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:45.970686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:45.970686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:45.970686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:45.970686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:45.972006 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:45.972006 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:45.972006 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:45.972006 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:45.972006 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 409/798] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:45.972297 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184c907-868c-48e7-9454-edd27d8adf95 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:45.981543 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4708b4-0266-4228-8960-0ae528a5cb9a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:46.027343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbc2a6-9427-4a09-824a-91418a191047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:46.028019 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbc2a6-9427-4a09-824a-91418a191047 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:46.028299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbc2a6-9427-4a09-824a-91418a191047 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:46.028571 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbc2a6-9427-4a09-824a-91418a191047 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:46.029741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fbc2a6-9427-4a09-824a-91418a191047 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:46.030049 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fbc2a6-9427-4a09-824a-91418a191047 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:46.031589 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fbc2a6-9427-4a09-824a-91418a191047 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:46.047421 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 390/799] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:46.047421 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbc2a6-9427-4a09-824a-91418a191047 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:46.055611 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:46.082627 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184c907-868c-48e7-9454-edd27d8adf95 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:46.118893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666c4f4-b8d8-4e84-81ff-4be983b45c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:46.119823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666c4f4-b8d8-4e84-81ff-4be983b45c94 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:46.119997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666c4f4-b8d8-4e84-81ff-4be983b45c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:46.120264 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666c4f4-b8d8-4e84-81ff-4be983b45c94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:46.131880 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666c4f4-b8d8-4e84-81ff-4be983b45c94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:46.133341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666c4f4-b8d8-4e84-81ff-4be983b45c94 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:46.152471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-d666c4f4-b8d8-4e84-81ff-4be983b45c94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:46.152471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:46.152471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:46.152471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:46.152471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:46.152471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:46.152471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:46.152471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:46.152471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:46.152471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:46.152471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:46.152471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:46.152471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:46.152471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:46.152471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:46.153459 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:46.153459 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:46.153459 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:46.153459 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:46.153459 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:46.153459 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:46.153459 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:46.153459 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:46.153459 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:46.153459 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:46.153459 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:46.153459 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:46.153459 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:46.153459 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:46.153459 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:46.153459 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:46.153459 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:46.154953 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:46.154953 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:46.154953 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:46.154953 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:46.155173 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbc2a6-9427-4a09-824a-91418a191047 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:46.155247 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 410/800] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:46.155496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e02b29-c3f3-48ba-b05b-7b1f12cdb584 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:46.155650 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e02b29-c3f3-48ba-b05b-7b1f12cdb584 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:46.155902 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e02b29-c3f3-48ba-b05b-7b1f12cdb584 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:46.159496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 411/801] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:46.159496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e02b29-c3f3-48ba-b05b-7b1f12cdb584 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:46.195085 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77995b-b828-4e13-b605-b9587014bac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:46.195845 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77995b-b828-4e13-b605-b9587014bac0 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:46.196238 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77995b-b828-4e13-b605-b9587014bac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:46.196520 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77995b-b828-4e13-b605-b9587014bac0 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:46.197248 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca77995b-b828-4e13-b605-b9587014bac0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:46.197557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca77995b-b828-4e13-b605-b9587014bac0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:46.197864 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:46.198925 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca77995b-b828-4e13-b605-b9587014bac0 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:46.216081 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 391/802] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:46.216081 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77995b-b828-4e13-b605-b9587014bac0 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:46.273865 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e02b29-c3f3-48ba-b05b-7b1f12cdb584 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:46.280201 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:46.308980 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177b552-40df-4faf-8a32-3bcd65e304d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:46.309688 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177b552-40df-4faf-8a32-3bcd65e304d9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:46.309981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177b552-40df-4faf-8a32-3bcd65e304d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:46.310252 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177b552-40df-4faf-8a32-3bcd65e304d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:46.316961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77995b-b828-4e13-b605-b9587014bac0 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:46.319327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177b552-40df-4faf-8a32-3bcd65e304d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:46.321334 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c177b552-40df-4faf-8a32-3bcd65e304d9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:46.336525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-c177b552-40df-4faf-8a32-3bcd65e304d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:46.336525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:46.336525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:46.336525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:46.336525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:46.336525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:46.336525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:46.336525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:46.336525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:46.336525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:46.336525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:46.336525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:46.336525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:14:46.336525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:14:46.336525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:14:46.337670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:14:46.337670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:46.337670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:46.337670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:46.337670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:46.337670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:46.337670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:46.337670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:46.337670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:46.337670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:46.337670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:46.337670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:46.337670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:46.337670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:46.337670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:46.337670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:46.337670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:46.343701 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:14:46.343701 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:14:46.343701 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:14:46.343701 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:14:46.343701 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:14:46.343701 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:14:46.343701 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:46.343701 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:46.343701 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 412/803] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:46.343701 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442dfb04-0533-403a-8c90-0a30883d38a3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:46.343701 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442dfb04-0533-403a-8c90-0a30883d38a3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:46.344457 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442dfb04-0533-403a-8c90-0a30883d38a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:46.359432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-442dfb04-0533-403a-8c90-0a30883d38a3 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:46.360057 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e947916-5807-4d69-8f83-9dc207a5a715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:46.360660 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-442dfb04-0533-403a-8c90-0a30883d38a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:46.361056 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e947916-5807-4d69-8f83-9dc207a5a715 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:46.361330 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e947916-5807-4d69-8f83-9dc207a5a715 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:46.361603 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e947916-5807-4d69-8f83-9dc207a5a715 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:46.362355 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e947916-5807-4d69-8f83-9dc207a5a715 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:46.362659 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e947916-5807-4d69-8f83-9dc207a5a715 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:46.364042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e947916-5807-4d69-8f83-9dc207a5a715 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:46.375120 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 392/804] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:46.376550 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e947916-5807-4d69-8f83-9dc207a5a715 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:46.427354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 413/805] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:14:46.428298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442dfb04-0533-403a-8c90-0a30883d38a3 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:46.452830 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:46.478054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e947916-5807-4d69-8f83-9dc207a5a715 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:46.504456 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:46.513770 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ec6c7f-4510-4912-86eb-015d6e446a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:46.514563 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec6c7f-4510-4912-86eb-015d6e446a19 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:46.514844 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec6c7f-4510-4912-86eb-015d6e446a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:46.515114 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ec6c7f-4510-4912-86eb-015d6e446a19 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:46.523500 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec6c7f-4510-4912-86eb-015d6e446a19 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:46.524943 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ec6c7f-4510-4912-86eb-015d6e446a19 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:46.527684 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442dfb04-0533-403a-8c90-0a30883d38a3 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:46.538557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-62ec6c7f-4510-4912-86eb-015d6e446a19 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 10:14:46.538557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:46.538557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:46.538557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:46.538557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:46.538557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:46.538557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:46.538557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:46.538557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:46.538557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:46.538557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:46.538557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:46.538557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:14:46.538557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:14:46.538557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:14:46.538557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:14:46.539623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:46.539623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:46.539623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:14:46.539623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:46.539623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:14:46.539623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:46.539623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:46.539623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:46.539623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:46.539623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:46.539623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:46.539623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:46.539623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:46.539623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:46.539623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:46.539623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:46.539623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:46.539623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:46.540627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:46.540627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:46.540627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:14:46.540627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:14:46.540627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:14:46.540627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:14:46.540627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:14:46.540627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:14:46.540627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 10:14:46.540627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:46.540627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 393/806] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jun 30 10:14:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:46.541362 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b5fca-5d01-4b57-a807-37d3c7afa1bd None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:46.541362 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b5fca-5d01-4b57-a807-37d3c7afa1bd None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:46.541494 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b5fca-5d01-4b57-a807-37d3c7afa1bd None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:46.542735 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 394/807] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:46.543447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b5fca-5d01-4b57-a807-37d3c7afa1bd None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:46.566981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1aa89-baa6-4ce9-b6a5-e653140fb2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:46.567614 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1aa89-baa6-4ce9-b6a5-e653140fb2b0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:46.567900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1aa89-baa6-4ce9-b6a5-e653140fb2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:46.568177 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c1aa89-baa6-4ce9-b6a5-e653140fb2b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:46.568972 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1aa89-baa6-4ce9-b6a5-e653140fb2b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:46.569286 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1aa89-baa6-4ce9-b6a5-e653140fb2b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:46.570736 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c1aa89-baa6-4ce9-b6a5-e653140fb2b0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:46.582901 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 414/808] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:14:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:46.583733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1aa89-baa6-4ce9-b6a5-e653140fb2b0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:46.647963 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b5fca-5d01-4b57-a807-37d3c7afa1bd None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:46.665621 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:46.681722 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c1aa89-baa6-4ce9-b6a5-e653140fb2b0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:46.682260 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1bc16f-7f48-4cd0-ab32-b5b289fac7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:46.685281 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1bc16f-7f48-4cd0-ab32-b5b289fac7fd None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:46.685281 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1bc16f-7f48-4cd0-ab32-b5b289fac7fd None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:46.685281 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1bc16f-7f48-4cd0-ab32-b5b289fac7fd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:46.692196 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1bc16f-7f48-4cd0-ab32-b5b289fac7fd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:46.693654 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1bc16f-7f48-4cd0-ab32-b5b289fac7fd None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:46.709080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-eb1bc16f-7f48-4cd0-ab32-b5b289fac7fd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 10:14:46.709080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:46.709080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:46.709080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:46.709080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:46.709080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:46.709080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:46.709080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:46.709080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:46.709080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:46.709080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:46.709080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:46.709080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:14:46.709080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:14:46.709080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:14:46.709080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:14:46.710020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:46.710020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:46.710020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:14:46.710020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:46.710020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:14:46.710020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:46.710020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:46.710020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:46.710020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:46.710020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:46.710020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:46.710020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:46.710020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:46.710020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:46.710020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:46.710020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:46.710020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:46.710020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:46.710958 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:46.710958 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:46.710958 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:14:46.710958 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:14:46.710958 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:14:46.710958 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:14:46.710958 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:14:46.710958 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:14:46.710958 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 10:14:46.710958 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:46.711594 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 395/809] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jun 30 10:14:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:46.712432 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1bc16f-7f48-4cd0-ab32-b5b289fac7fd None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:46.715411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9b337c-9587-40ed-9391-37f0a503f04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:46.716452 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b337c-9587-40ed-9391-37f0a503f04d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:46.716906 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b337c-9587-40ed-9391-37f0a503f04d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:46.717580 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9b337c-9587-40ed-9391-37f0a503f04d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:46.718234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b337c-9587-40ed-9391-37f0a503f04d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:46.719690 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9b337c-9587-40ed-9391-37f0a503f04d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:46.732581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-7c9b337c-9587-40ed-9391-37f0a503f04d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 10:14:46.732581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:46.732581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 10:14:46.732581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 10:14:46.732581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 10:14:46.732581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 10:14:46.732581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 10:14:46.732581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:14:46.732581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 10:14:46.732581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:46.732581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:14:46.732581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:46.732581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:46.732581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:46.732581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:46.732581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:46.738429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:46.738429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:46.738429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:46.738429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:46.738429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:46.738429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:46.738429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:46.738429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 10:14:46.738429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 10:14:46.738429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 10:14:46.738429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 10:14:46.738429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:46.738429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:46.738429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:46.738429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:46.738429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:46.739780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:46.739780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:46.739780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:46.739780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:46.739780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:46.739780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:46.739780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:46.739780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:46.739780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:46.739780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:46.739780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:46.739780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 10:14:46.739780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:14:46.739780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 30 10:14:46.739780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:46.739780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 415/810] 10.222.0.47 () {56 vars in 1129 bytes} [Thu Jun 30 10:14:46 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:46.741093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533540be-7183-4416-9dd4-2ac3a7ef86c0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:46.741093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533540be-7183-4416-9dd4-2ac3a7ef86c0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:46.741093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533540be-7183-4416-9dd4-2ac3a7ef86c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:46.752366 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-533540be-7183-4416-9dd4-2ac3a7ef86c0 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:46.753539 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-533540be-7183-4416-9dd4-2ac3a7ef86c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:46.764200 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:46.817345 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1bc16f-7f48-4cd0-ab32-b5b289fac7fd None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:46.820209 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 416/811] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:14:46.821001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533540be-7183-4416-9dd4-2ac3a7ef86c0 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:46.861095 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6dfa2d-0c12-4b3d-bfa7-85e37a9d1ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:46.861773 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6dfa2d-0c12-4b3d-bfa7-85e37a9d1ae1 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:46.862060 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6dfa2d-0c12-4b3d-bfa7-85e37a9d1ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:46.862341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6dfa2d-0c12-4b3d-bfa7-85e37a9d1ae1 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:46.863192 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6dfa2d-0c12-4b3d-bfa7-85e37a9d1ae1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:46.863540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6dfa2d-0c12-4b3d-bfa7-85e37a9d1ae1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:46.865240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6dfa2d-0c12-4b3d-bfa7-85e37a9d1ae1 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:46.881888 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 396/812] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jun 30 10:14:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:46.882692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6dfa2d-0c12-4b3d-bfa7-85e37a9d1ae1 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:46.906208 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:46.927390 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533540be-7183-4416-9dd4-2ac3a7ef86c0 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:46.962185 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d47c6-2a9c-4759-8535-14460a2cc177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:46.963060 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d47c6-2a9c-4759-8535-14460a2cc177 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:46.963369 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d47c6-2a9c-4759-8535-14460a2cc177 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:46.963645 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1d47c6-2a9c-4759-8535-14460a2cc177 None admin] PATH_INFO: `/v3/projects/50902e0539434c82b505dc5009aeaf85/users/07de926cb8594b098f79111dab8a9cc6/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:46.965572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1d47c6-2a9c-4759-8535-14460a2cc177 None admin] RBAC: Authorizing `identity:create_grant(project_id=50902e0539434c82b505dc5009aeaf85, user_id=07de926cb8594b098f79111dab8a9cc6, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:46.966999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1d47c6-2a9c-4759-8535-14460a2cc177 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:46.980847 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6dfa2d-0c12-4b3d-bfa7-85e37a9d1ae1 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:46.983134 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:47.013299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93463fb6-1e47-4fb4-bc98-37628a750574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:47.013959 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93463fb6-1e47-4fb4-bc98-37628a750574 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:47.014240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93463fb6-1e47-4fb4-bc98-37628a750574 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:47.014514 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93463fb6-1e47-4fb4-bc98-37628a750574 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:47.015458 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93463fb6-1e47-4fb4-bc98-37628a750574 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:47.015767 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93463fb6-1e47-4fb4-bc98-37628a750574 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:47.018224 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93463fb6-1e47-4fb4-bc98-37628a750574 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:47.028486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 417/813] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jun 30 10:14:46 2022] PUT /identity/v3/projects/50902e0539434c82b505dc5009aeaf85/users/07de926cb8594b098f79111dab8a9cc6/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:14:47.031970 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d47c6-2a9c-4759-8535-14460a2cc177 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:47.036851 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 397/814] 10.222.0.47 () {56 vars in 1149 bytes} [Thu Jun 30 10:14:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:47.037507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93463fb6-1e47-4fb4-bc98-37628a750574 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:47.153616 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93463fb6-1e47-4fb4-bc98-37628a750574 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:47.158914 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1d47c6-2a9c-4759-8535-14460a2cc177 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:47.198573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1856e-5b5b-4bdf-94ae-c92d4607831c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:47.199488 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1856e-5b5b-4bdf-94ae-c92d4607831c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:47.199782 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1856e-5b5b-4bdf-94ae-c92d4607831c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:47.200065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1856e-5b5b-4bdf-94ae-c92d4607831c None admin] PATH_INFO: `/v3/services/network` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:47.200738 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f1856e-5b5b-4bdf-94ae-c92d4607831c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:47.202117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f1856e-5b5b-4bdf-94ae-c92d4607831c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:47.202913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d60d2-0bbd-4a17-a2bd-3a963e68a237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:47.203614 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d60d2-0bbd-4a17-a2bd-3a963e68a237 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:47.204019 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d60d2-0bbd-4a17-a2bd-3a963e68a237 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:47.204902 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d60d2-0bbd-4a17-a2bd-3a963e68a237 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:47.205616 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d60d2-0bbd-4a17-a2bd-3a963e68a237 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:47.205964 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d60d2-0bbd-4a17-a2bd-3a963e68a237 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:47.207353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162d60d2-0bbd-4a17-a2bd-3a963e68a237 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:47.212021 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 418/815] 10.222.0.47 () {56 vars in 1124 bytes} [Thu Jun 30 10:14:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:47.212891 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d60d2-0bbd-4a17-a2bd-3a963e68a237 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:47.216930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-55f1856e-5b5b-4bdf-94ae-c92d4607831c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 10:14:47.216930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:47.216930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 10:14:47.216930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 10:14:47.216930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 10:14:47.216930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 10:14:47.216930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 10:14:47.216930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:14:47.216930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 10:14:47.216930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:47.216930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:14:47.216930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:47.216930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:47.216930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:47.216930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:47.216930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:47.216930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:47.217918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:47.217918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:47.217918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:47.217918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:47.217918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:47.217918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:47.217918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 10:14:47.217918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 10:14:47.217918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 10:14:47.217918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 10:14:47.217918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:47.217918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:47.217918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:47.217918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:47.217918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:47.217918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:47.218791 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:47.218791 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:47.218791 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:47.218791 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:47.218791 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:47.218791 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:47.218791 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:47.218791 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:47.218791 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:47.218791 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:47.218791 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 10:14:47.218791 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:14:47.218791 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 30 10:14:47.218791 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:47.219616 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:47.221333 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 398/816] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:47 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:47.221333 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1856e-5b5b-4bdf-94ae-c92d4607831c None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:47.253640 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:47.328800 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d60d2-0bbd-4a17-a2bd-3a963e68a237 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:47.331527 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1856e-5b5b-4bdf-94ae-c92d4607831c None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:47.364407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7033e-0cea-4cf4-b5ad-0ffeed821658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:47.365119 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7033e-0cea-4cf4-b5ad-0ffeed821658 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:47.365347 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7033e-0cea-4cf4-b5ad-0ffeed821658 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:47.365624 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7033e-0cea-4cf4-b5ad-0ffeed821658 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:47.366741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d3831-8ace-4119-9fa2-38e0db526c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:47.367571 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d3831-8ace-4119-9fa2-38e0db526c6c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:47.367862 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d3831-8ace-4119-9fa2-38e0db526c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:47.368152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d3831-8ace-4119-9fa2-38e0db526c6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:47.379772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425d3831-8ace-4119-9fa2-38e0db526c6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:47.381258 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425d3831-8ace-4119-9fa2-38e0db526c6c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:47.401029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-425d3831-8ace-4119-9fa2-38e0db526c6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:47.401029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:47.401029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:47.401029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:47.401029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:47.401029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:47.401029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:47.401029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:47.401029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:47.401029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:47.401029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:47.401029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:47.401029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:47.401029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:47.401029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:47.401955 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:47.401955 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:47.401955 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:47.401955 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:47.401955 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:47.401955 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:47.401955 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:47.401955 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:47.401955 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:47.401955 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:47.401955 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:47.401955 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:47.401955 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:47.401955 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:47.401955 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:47.401955 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:47.401955 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:47.402849 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:47.402849 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:47.402849 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:47.402849 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:47.402849 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 399/817] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:47.403135 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d3831-8ace-4119-9fa2-38e0db526c6c None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:47.441855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe7033e-0cea-4cf4-b5ad-0ffeed821658 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:47.443364 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe7033e-0cea-4cf4-b5ad-0ffeed821658 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:47.445283 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:47.493171 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:47.517760 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d3831-8ace-4119-9fa2-38e0db526c6c None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:47.556921 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57dfe3d-4bc8-4c85-b910-bce53322348c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:47.557593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dfe3d-4bc8-4c85-b910-bce53322348c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:47.557880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dfe3d-4bc8-4c85-b910-bce53322348c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:47.558410 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57dfe3d-4bc8-4c85-b910-bce53322348c None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:47.558866 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57dfe3d-4bc8-4c85-b910-bce53322348c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:47.559174 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57dfe3d-4bc8-4c85-b910-bce53322348c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:47.560572 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57dfe3d-4bc8-4c85-b910-bce53322348c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:47.566143 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 400/818] 10.222.0.47 () {56 vars in 1124 bytes} [Thu Jun 30 10:14:47 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:47.571131 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57dfe3d-4bc8-4c85-b910-bce53322348c None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:47.571671 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 419/819] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:14:47 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 10:14:47.572291 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7033e-0cea-4cf4-b5ad-0ffeed821658 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:47.677859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57dfe3d-4bc8-4c85-b910-bce53322348c None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:47.678061 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7033e-0cea-4cf4-b5ad-0ffeed821658 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:47.679506 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:47.713608 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae997eb-f522-4cd9-9f08-f6c8730a32c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:47.714315 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae997eb-f522-4cd9-9f08-f6c8730a32c9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:47.714609 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae997eb-f522-4cd9-9f08-f6c8730a32c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:47.714890 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae997eb-f522-4cd9-9f08-f6c8730a32c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:47.715655 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae997eb-f522-4cd9-9f08-f6c8730a32c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:47.715859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210323d-814c-4c38-a883-5f9bab4ab3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:47.715987 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae997eb-f522-4cd9-9f08-f6c8730a32c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:47.716532 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210323d-814c-4c38-a883-5f9bab4ab3d7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:47.716820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210323d-814c-4c38-a883-5f9bab4ab3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:47.717100 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d210323d-814c-4c38-a883-5f9bab4ab3d7 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:47.717335 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae997eb-f522-4cd9-9f08-f6c8730a32c9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:47.717807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d210323d-814c-4c38-a883-5f9bab4ab3d7 None admin] RBAC: Adding query filter params (name=network) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:47.718162 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d210323d-814c-4c38-a883-5f9bab4ab3d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:47.720602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d210323d-814c-4c38-a883-5f9bab4ab3d7 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:47.733956 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 401/820] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jun 30 10:14:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:47.733956 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210323d-814c-4c38-a883-5f9bab4ab3d7 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:47.734224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 420/821] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:47.734224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae997eb-f522-4cd9-9f08-f6c8730a32c9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:47.742605 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:47.838742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d210323d-814c-4c38-a883-5f9bab4ab3d7 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:47.840472 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae997eb-f522-4cd9-9f08-f6c8730a32c9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:47.874382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7417a-0f59-4299-a342-22d55c250f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:47.875698 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7417a-0f59-4299-a342-22d55c250f1b None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:47.875995 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7417a-0f59-4299-a342-22d55c250f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:47.876460 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf7417a-0f59-4299-a342-22d55c250f1b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:47.877173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719eec2d-dc11-4bc2-83d1-185ed27e16aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:47.877289 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf7417a-0f59-4299-a342-22d55c250f1b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:47.877966 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719eec2d-dc11-4bc2-83d1-185ed27e16aa None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:47.878249 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719eec2d-dc11-4bc2-83d1-185ed27e16aa None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:47.878526 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719eec2d-dc11-4bc2-83d1-185ed27e16aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:47.878621 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf7417a-0f59-4299-a342-22d55c250f1b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:47.889664 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719eec2d-dc11-4bc2-83d1-185ed27e16aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:47.891113 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719eec2d-dc11-4bc2-83d1-185ed27e16aa None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:47.909480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-719eec2d-dc11-4bc2-83d1-185ed27e16aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:47.909480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:47.909480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:47.909480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:47.909480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:47.909480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:47.909480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:47.909480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:47.909480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:47.909480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:47.909480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:47.909480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:47.909480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:47.909480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:47.909480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:47.910382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:47.910382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:47.910382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:47.910382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:47.910382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:47.910382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:47.910382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:47.910382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:47.910382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:47.910382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:47.910382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:47.910382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:47.910382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:47.910382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:47.910382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:47.910382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:47.910382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:47.911695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:47.911695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:47.911695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:47.911695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:47.911695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 421/822] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:47.911695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719eec2d-dc11-4bc2-83d1-185ed27e16aa None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:47.928203 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:47.973445 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 402/823] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jun 30 10:14:47 2022] POST /identity/v3/endpoints => generated 346 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:14:47.974546 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7417a-0f59-4299-a342-22d55c250f1b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:48.032810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719eec2d-dc11-4bc2-83d1-185ed27e16aa None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:48.066913 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:48.079046 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d68d7-3b4e-49d3-85bd-7df6d727afec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:48.080251 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d68d7-3b4e-49d3-85bd-7df6d727afec None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:48.080742 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d68d7-3b4e-49d3-85bd-7df6d727afec None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:48.081166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d68d7-3b4e-49d3-85bd-7df6d727afec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:48.089567 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d68d7-3b4e-49d3-85bd-7df6d727afec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:48.091463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d68d7-3b4e-49d3-85bd-7df6d727afec None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:48.107601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-745d68d7-3b4e-49d3-85bd-7df6d727afec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:48.107601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:48.107601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:48.107601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:48.107601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:48.107601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:48.107601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:48.107601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:48.107601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:48.107601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:48.107601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:48.107601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:48.107601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:14:48.107601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:14:48.107601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:14:48.108733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:14:48.108733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:48.108733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:48.108733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:48.108733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:48.108733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:48.108733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:48.108733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:48.108733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:48.108733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:48.108733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:48.108733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:48.108733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:48.108733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:48.108733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:48.108733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:48.108733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:48.109861 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:14:48.109861 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:14:48.109861 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:14:48.109861 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:14:48.109861 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:14:48.109861 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:14:48.109861 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:48.109861 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:48.112694 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 422/824] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:48.113790 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d68d7-3b4e-49d3-85bd-7df6d727afec None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:48.125506 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf7417a-0f59-4299-a342-22d55c250f1b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:48.181315 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c8cda-b0ad-4566-ac29-a0bbab4fd9f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:48.182995 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c8cda-b0ad-4566-ac29-a0bbab4fd9f1 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:48.183432 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c8cda-b0ad-4566-ac29-a0bbab4fd9f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:48.183820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c8cda-b0ad-4566-ac29-a0bbab4fd9f1 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:48.184627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c8cda-b0ad-4566-ac29-a0bbab4fd9f1 None admin] RBAC: Adding query filter params (type=network) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:48.185000 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c8cda-b0ad-4566-ac29-a0bbab4fd9f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:48.192542 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c8cda-b0ad-4566-ac29-a0bbab4fd9f1 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:48.193102 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 403/825] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jun 30 10:14:47 2022] GET /identity/v3/services?type=network => generated 350 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:48.194005 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c8cda-b0ad-4566-ac29-a0bbab4fd9f1 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:48.213300 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:48.257739 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d68d7-3b4e-49d3-85bd-7df6d727afec None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:48.304729 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aaf416-e913-4df3-84a7-7a50489df508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:48.306086 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aaf416-e913-4df3-84a7-7a50489df508 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:48.306780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aaf416-e913-4df3-84a7-7a50489df508 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:48.307199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aaf416-e913-4df3-84a7-7a50489df508 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:48.308351 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aaf416-e913-4df3-84a7-7a50489df508 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:48.309220 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aaf416-e913-4df3-84a7-7a50489df508 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:48.311291 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aaf416-e913-4df3-84a7-7a50489df508 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:48.315876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c8cda-b0ad-4566-ac29-a0bbab4fd9f1 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:48.328723 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 423/826] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:48.331071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aaf416-e913-4df3-84a7-7a50489df508 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:48.346352 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:48.357912 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c8f72-f4fc-45a5-bf2a-5ebd71b0105f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:48.358627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c8f72-f4fc-45a5-bf2a-5ebd71b0105f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:48.358978 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c8f72-f4fc-45a5-bf2a-5ebd71b0105f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:48.359353 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c8f72-f4fc-45a5-bf2a-5ebd71b0105f None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:48.360897 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c8f72-f4fc-45a5-bf2a-5ebd71b0105f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:48.362889 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c8f72-f4fc-45a5-bf2a-5ebd71b0105f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:48.363006 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c8f72-f4fc-45a5-bf2a-5ebd71b0105f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:48.377109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 404/827] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:14:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:48.378248 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c8f72-f4fc-45a5-bf2a-5ebd71b0105f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:48.468989 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aaf416-e913-4df3-84a7-7a50489df508 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:48.503455 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c8f72-f4fc-45a5-bf2a-5ebd71b0105f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:48.504082 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:48.515311 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799686e8-80e7-4971-b697-b88004320026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:48.515311 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799686e8-80e7-4971-b697-b88004320026 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:48.515311 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799686e8-80e7-4971-b697-b88004320026 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:48.515311 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799686e8-80e7-4971-b697-b88004320026 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:48.518820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799686e8-80e7-4971-b697-b88004320026 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:48.518820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799686e8-80e7-4971-b697-b88004320026 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:48.546277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eaa149-67f7-46bb-b2fd-ba89eda9fc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:48.547100 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eaa149-67f7-46bb-b2fd-ba89eda9fc92 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:48.547403 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eaa149-67f7-46bb-b2fd-ba89eda9fc92 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:48.547696 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eaa149-67f7-46bb-b2fd-ba89eda9fc92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:48.554600 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eaa149-67f7-46bb-b2fd-ba89eda9fc92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:48.556106 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eaa149-67f7-46bb-b2fd-ba89eda9fc92 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:48.570225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-f6eaa149-67f7-46bb-b2fd-ba89eda9fc92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:48.570225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:48.570225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:48.570225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:48.570225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:48.570225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:48.570225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:48.570225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:48.570225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:48.570225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:48.570225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:48.570225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:48.570225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:14:48.570225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:14:48.570225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:14:48.571167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:14:48.571167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:48.571167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:48.571167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:48.571167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:48.571167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:48.571167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:48.571167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:48.571167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:48.571167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:48.571167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:48.571167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:48.571167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:48.571167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:48.571167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:48.571167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:48.571167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:48.572132 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:14:48.572132 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:14:48.572132 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:14:48.572132 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:14:48.572132 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:14:48.572132 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:14:48.572132 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:48.572132 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:48.572132 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 405/828] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:48.572634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eaa149-67f7-46bb-b2fd-ba89eda9fc92 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:48.598766 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 424/829] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jun 30 10:14:48 2022] POST /identity/v3/endpoints => generated 327 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:14:48.600378 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799686e8-80e7-4971-b697-b88004320026 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:48.638358 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:48.735314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eaa149-67f7-46bb-b2fd-ba89eda9fc92 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:48.807418 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799686e8-80e7-4971-b697-b88004320026 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:48.809416 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496ef40-865d-46de-a00e-49136729b983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:48.810600 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496ef40-865d-46de-a00e-49136729b983 None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:48.810882 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496ef40-865d-46de-a00e-49136729b983 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:48.811156 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496ef40-865d-46de-a00e-49136729b983 None admin] PATH_INFO: `/v3/projects/50902e0539434c82b505dc5009aeaf85/users/66ad8defb9b6453c96f1a8a7343695ba/roles/99d8cd9cd87e472290bf0de2710b5857` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:48.835394 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a496ef40-865d-46de-a00e-49136729b983 None admin] RBAC: Authorizing `identity:create_grant(project_id=50902e0539434c82b505dc5009aeaf85, user_id=66ad8defb9b6453c96f1a8a7343695ba, role_id=99d8cd9cd87e472290bf0de2710b5857)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:48.837421 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a496ef40-865d-46de-a00e-49136729b983 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:48.855984 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c270311-f348-48f6-9c15-63f13bd9083d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:48.857020 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c270311-f348-48f6-9c15-63f13bd9083d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:48.857403 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c270311-f348-48f6-9c15-63f13bd9083d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:48.857818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c270311-f348-48f6-9c15-63f13bd9083d None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:48.859586 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c270311-f348-48f6-9c15-63f13bd9083d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:48.860002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c270311-f348-48f6-9c15-63f13bd9083d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:48.862222 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c270311-f348-48f6-9c15-63f13bd9083d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:48.875229 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 425/830] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:14:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:48.881072 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c270311-f348-48f6-9c15-63f13bd9083d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:48.931422 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 406/831] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jun 30 10:14:48 2022] PUT /identity/v3/projects/50902e0539434c82b505dc5009aeaf85/users/66ad8defb9b6453c96f1a8a7343695ba/roles/99d8cd9cd87e472290bf0de2710b5857 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:14:48.935402 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:49.006647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c270311-f348-48f6-9c15-63f13bd9083d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:49.048558 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17a115-fa05-40ff-9c88-9c4fab73da81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:49.049484 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17a115-fa05-40ff-9c88-9c4fab73da81 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:49.049875 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17a115-fa05-40ff-9c88-9c4fab73da81 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:49.050253 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17a115-fa05-40ff-9c88-9c4fab73da81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:49.051208 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d17a115-fa05-40ff-9c88-9c4fab73da81 None admin] RBAC: Adding query filter params (role.id=e63bda042e104a4a93fba15ffcfe1183, scope.project.id=50902e0539434c82b505dc5009aeaf85, user.id=71f6d7715e6f49ffb9fdf6e1307bbd51) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:49.051696 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d17a115-fa05-40ff-9c88-9c4fab73da81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:49.053268 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d17a115-fa05-40ff-9c88-9c4fab73da81 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:49.060817 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 426/832] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:14:48 2022] GET /identity/v3/role_assignments?user.id=71f6d7715e6f49ffb9fdf6e1307bbd51&scope.project.id=50902e0539434c82b505dc5009aeaf85&role.id=e63bda042e104a4a93fba15ffcfe1183 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:14:49.113949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7662fc-f309-4f8e-b163-50200d414dd8 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:49.114237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7662fc-f309-4f8e-b163-50200d414dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:49.114530 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7662fc-f309-4f8e-b163-50200d414dd8 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:49.116321 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 407/833] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:49.127731 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dde835-a371-47b6-aac1-a11dd39cd599 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:49.128165 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dde835-a371-47b6-aac1-a11dd39cd599 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:49.128454 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dde835-a371-47b6-aac1-a11dd39cd599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:49.147214 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c7dde835-a371-47b6-aac1-a11dd39cd599 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:49.148657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c7dde835-a371-47b6-aac1-a11dd39cd599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:49.227142 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 427/834] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:49 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:14:49.251864 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7662fc-f309-4f8e-b163-50200d414dd8 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:49.350793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:49.360962 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7662fc-f309-4f8e-b163-50200d414dd8 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:49.398047 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c5eab-f9f9-4aab-b883-a48fe7d1734b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:49.398933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c5eab-f9f9-4aab-b883-a48fe7d1734b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:49.399307 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c5eab-f9f9-4aab-b883-a48fe7d1734b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:49.399673 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70c5eab-f9f9-4aab-b883-a48fe7d1734b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:49.410815 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70c5eab-f9f9-4aab-b883-a48fe7d1734b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:49.424646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-d70c5eab-f9f9-4aab-b883-a48fe7d1734b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:14:49.424646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:49.424646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:49.424646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:49.424646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:49.424646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:49.424646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:49.424646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:49.424646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:49.424646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:49.424646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:49.424646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:49.424646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:49.424646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:49.424646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:49.424646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:49.425812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:49.425812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:49.425812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:49.425812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:49.425812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:49.425812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:49.425812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:49.425812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:49.425812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:49.425812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:49.425812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:49.425812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:49.425812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:49.425812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:49.425812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:49.425812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:49.426872 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:49.426872 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:49.426872 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:49.426872 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:49.426872 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:49.426872 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:49.426872 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:49.426872 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:49.426872 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:49.426872 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:49.426872 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:14:49.426872 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:49.427961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 408/835] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:14:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:49.434848 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dde835-a371-47b6-aac1-a11dd39cd599 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:49.472204 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:49.549274 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dde835-a371-47b6-aac1-a11dd39cd599 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:49.586646 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abb04b-78dc-4bda-b046-f2cacc8c2040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:49.587726 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abb04b-78dc-4bda-b046-f2cacc8c2040 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:49.588211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abb04b-78dc-4bda-b046-f2cacc8c2040 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:49.588713 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0abb04b-78dc-4bda-b046-f2cacc8c2040 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:49.589680 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0abb04b-78dc-4bda-b046-f2cacc8c2040 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:49.590186 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0abb04b-78dc-4bda-b046-f2cacc8c2040 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:49.591935 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0abb04b-78dc-4bda-b046-f2cacc8c2040 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:49.602861 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 428/836] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:14:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:49.608718 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c5eab-f9f9-4aab-b883-a48fe7d1734b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:49.715660 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70c5eab-f9f9-4aab-b883-a48fe7d1734b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:49.750983 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22670f0-19c3-4551-adad-3e3379e09092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:49.751857 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22670f0-19c3-4551-adad-3e3379e09092 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:49.752209 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22670f0-19c3-4551-adad-3e3379e09092 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:49.752498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22670f0-19c3-4551-adad-3e3379e09092 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:49.763630 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22670f0-19c3-4551-adad-3e3379e09092 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:49.765094 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22670f0-19c3-4551-adad-3e3379e09092 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:49.784710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-b22670f0-19c3-4551-adad-3e3379e09092 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:49.784710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:49.784710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:49.784710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:49.784710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:49.784710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:49.784710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:49.784710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:49.784710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:49.784710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:49.784710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:49.784710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:49.784710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:49.784710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:49.784710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:49.785656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:49.785656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:49.785656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:49.785656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:49.785656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:49.785656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:49.785656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:49.785656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:49.785656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:49.785656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:49.785656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:49.785656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:49.785656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:49.785656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:49.785656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:49.785656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:49.785656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:49.786559 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:49.786559 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:49.786559 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:49.786559 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:49.786559 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 409/837] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:49.792939 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abb04b-78dc-4bda-b046-f2cacc8c2040 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:49.893758 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:49.897690 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0abb04b-78dc-4bda-b046-f2cacc8c2040 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:49.932058 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161dda4c-a40e-44d7-a6b2-c1877cc2dfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:49.932727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161dda4c-a40e-44d7-a6b2-c1877cc2dfb2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:49.933009 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161dda4c-a40e-44d7-a6b2-c1877cc2dfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:49.933282 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161dda4c-a40e-44d7-a6b2-c1877cc2dfb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:49.934009 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161dda4c-a40e-44d7-a6b2-c1877cc2dfb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:49.934314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161dda4c-a40e-44d7-a6b2-c1877cc2dfb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:49.935727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161dda4c-a40e-44d7-a6b2-c1877cc2dfb2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:49.951626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 429/838] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:49.953010 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22670f0-19c3-4551-adad-3e3379e09092 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:49.971610 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:50.060123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22670f0-19c3-4551-adad-3e3379e09092 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:50.095811 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc73cf8-a269-4588-a3fd-0fcf556af42c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:50.096623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc73cf8-a269-4588-a3fd-0fcf556af42c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:50.096913 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc73cf8-a269-4588-a3fd-0fcf556af42c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:50.097198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc73cf8-a269-4588-a3fd-0fcf556af42c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:50.105199 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc73cf8-a269-4588-a3fd-0fcf556af42c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:50.106664 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc73cf8-a269-4588-a3fd-0fcf556af42c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:50.121308 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-cdc73cf8-a269-4588-a3fd-0fcf556af42c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 30 10:14:50.121308 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:50.121308 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:50.121308 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:50.121308 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:50.121308 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:50.121308 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:50.121308 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:50.121308 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:50.121308 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:50.121308 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:50.121308 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:50.121308 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:14:50.121308 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:14:50.121308 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:14:50.121308 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:14:50.122263 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:50.122263 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:50.122263 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:14:50.122263 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:50.122263 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:14:50.122263 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:50.122263 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:50.122263 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:50.122263 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:50.122263 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:50.122263 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:50.122263 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:50.122263 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:50.122263 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:50.122263 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:50.122263 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:50.122263 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:50.122263 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:50.123224 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:50.123224 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:50.123224 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:14:50.123224 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:14:50.123224 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:14:50.123224 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:14:50.123224 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:14:50.123224 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:14:50.123224 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 30 10:14:50.123224 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:50.123224 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 410/839] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:14:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:50.128951 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161dda4c-a40e-44d7-a6b2-c1877cc2dfb2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:50.207539 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c49a9-3841-44fd-82d6-9a6aec938a8d None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:50.207904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c49a9-3841-44fd-82d6-9a6aec938a8d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:50.208108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214c49a9-3841-44fd-82d6-9a6aec938a8d None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:50.210766 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 411/840] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:14:50.214877 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcbfcc-250b-4f82-ad45-d5fd58e1a132 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:50.215171 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcbfcc-250b-4f82-ad45-d5fd58e1a132 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:50.215490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcbfcc-250b-4f82-ad45-d5fd58e1a132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:50.233943 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-fbdcbfcc-250b-4f82-ad45-d5fd58e1a132 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:50.235121 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-fbdcbfcc-250b-4f82-ad45-d5fd58e1a132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:50.237525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161dda4c-a40e-44d7-a6b2-c1877cc2dfb2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:50.271352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a17e8-274f-4aa0-b4d7-c7c093dd316b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:50.272322 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a17e8-274f-4aa0-b4d7-c7c093dd316b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:50.272605 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a17e8-274f-4aa0-b4d7-c7c093dd316b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:50.272881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630a17e8-274f-4aa0-b4d7-c7c093dd316b None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:50.273724 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a17e8-274f-4aa0-b4d7-c7c093dd316b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:50.274030 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a17e8-274f-4aa0-b4d7-c7c093dd316b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:50.275504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630a17e8-274f-4aa0-b4d7-c7c093dd316b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:50.295947 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 430/841] 10.222.0.47 () {56 vars in 1135 bytes} [Thu Jun 30 10:14:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:50.302265 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a17e8-274f-4aa0-b4d7-c7c093dd316b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:50.327749 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:50.417484 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630a17e8-274f-4aa0-b4d7-c7c093dd316b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:50.423517 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:50.433432 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 412/842] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:50 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 10:14:50.458074 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11bf2f-efdd-4f70-b328-079632a6da23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:50.458654 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11bf2f-efdd-4f70-b328-079632a6da23 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:50.458759 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdcbfcc-250b-4f82-ad45-d5fd58e1a132 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:50.458952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11bf2f-efdd-4f70-b328-079632a6da23 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:50.459205 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd11bf2f-efdd-4f70-b328-079632a6da23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:50.540395 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd11bf2f-efdd-4f70-b328-079632a6da23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:50.542523 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd11bf2f-efdd-4f70-b328-079632a6da23 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:50.564566 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:50.587098 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdcbfcc-250b-4f82-ad45-d5fd58e1a132 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:50.630809 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9e954-f6a6-440f-93b2-97a23914dd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:50.631992 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9e954-f6a6-440f-93b2-97a23914dd93 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:50.632277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9e954-f6a6-440f-93b2-97a23914dd93 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:50.632556 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af9e954-f6a6-440f-93b2-97a23914dd93 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:50.633214 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9e954-f6a6-440f-93b2-97a23914dd93 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:50.635179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af9e954-f6a6-440f-93b2-97a23914dd93 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:50.651039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-1af9e954-f6a6-440f-93b2-97a23914dd93 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 10:14:50.651039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:50.651039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 10:14:50.651039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 10:14:50.651039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 10:14:50.651039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 10:14:50.651039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 10:14:50.651039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:14:50.651039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 10:14:50.651039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:50.651039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:14:50.651039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:50.651039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:50.651039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:50.651039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:50.651039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:50.651039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:50.652233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:50.652233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:50.652233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:50.652233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:50.652233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:50.652233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:50.652233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 10:14:50.652233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 10:14:50.652233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 10:14:50.652233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 10:14:50.652233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:50.652233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:50.652233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:50.652233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:50.652233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:50.652233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:50.658062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:50.658062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:50.658062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:50.658062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:50.658062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:50.658062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:50.658062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:50.658062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:50.658062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:50.658062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:50.658062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 10:14:50.658062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:14:50.658062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 10:14:50.658062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:50.658062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 413/843] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jun 30 10:14:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:14:50.660528 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9e954-f6a6-440f-93b2-97a23914dd93 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:50.664562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 431/844] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:14:50 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 10:14:50.672260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11bf2f-efdd-4f70-b328-079632a6da23 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:50.739419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:50.780999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd11bf2f-efdd-4f70-b328-079632a6da23 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:50.796714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af9e954-f6a6-440f-93b2-97a23914dd93 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:50.799126 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:50.818860 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4e4d7-eb87-45e9-bc6b-3cbb2daeee52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:50.820047 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4e4d7-eb87-45e9-bc6b-3cbb2daeee52 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:50.820566 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4e4d7-eb87-45e9-bc6b-3cbb2daeee52 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:50.821048 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e4e4d7-eb87-45e9-bc6b-3cbb2daeee52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:50.835020 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4e4d7-eb87-45e9-bc6b-3cbb2daeee52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:50.837013 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e4e4d7-eb87-45e9-bc6b-3cbb2daeee52 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:50.838029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c9b86-05ed-437b-99ae-329cc46bfed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:50.838701 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c9b86-05ed-437b-99ae-329cc46bfed1 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:50.838983 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c9b86-05ed-437b-99ae-329cc46bfed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:50.839255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c9b86-05ed-437b-99ae-329cc46bfed1 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:50.840025 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c9b86-05ed-437b-99ae-329cc46bfed1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:50.840284 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c9b86-05ed-437b-99ae-329cc46bfed1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:50.841625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c9b86-05ed-437b-99ae-329cc46bfed1 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:50.847647 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 414/845] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jun 30 10:14:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:14:50.853102 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c9b86-05ed-437b-99ae-329cc46bfed1 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:50.865354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-83e4e4d7-eb87-45e9-bc6b-3cbb2daeee52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:50.865354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:50.865354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:50.865354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:50.865354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:50.865354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:50.865354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:50.865354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:50.865354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:50.865354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:50.865354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:50.865354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:50.865354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:50.865354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:50.865354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:50.866339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:50.866339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:50.866339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:50.866339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:50.866339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:50.866339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:50.866339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:50.866339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:50.866339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:50.866339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:50.866339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:50.866339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:50.866339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:50.866339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:50.866339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:50.866339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:50.866339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:50.867713 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:50.867713 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:50.867713 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:50.867713 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:50.871339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 432/846] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:50.872414 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4e4d7-eb87-45e9-bc6b-3cbb2daeee52 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:50.964858 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c9b86-05ed-437b-99ae-329cc46bfed1 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:50.980547 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e4e4d7-eb87-45e9-bc6b-3cbb2daeee52 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:50.990385 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:51.002967 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41421fa-d9d7-4177-9500-e3d35247a42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:51.003931 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41421fa-d9d7-4177-9500-e3d35247a42d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:51.004411 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41421fa-d9d7-4177-9500-e3d35247a42d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:51.004880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41421fa-d9d7-4177-9500-e3d35247a42d None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:51.005772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41421fa-d9d7-4177-9500-e3d35247a42d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:51.006289 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41421fa-d9d7-4177-9500-e3d35247a42d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:51.007866 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41421fa-d9d7-4177-9500-e3d35247a42d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:51.013023 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 415/847] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jun 30 10:14:50 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:51.018835 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addbe983-4fe1-4f74-8c16-f8e78ffdb04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:51.019917 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbe983-4fe1-4f74-8c16-f8e78ffdb04b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:51.020594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbe983-4fe1-4f74-8c16-f8e78ffdb04b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:51.021080 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addbe983-4fe1-4f74-8c16-f8e78ffdb04b None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:51.022308 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addbe983-4fe1-4f74-8c16-f8e78ffdb04b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:51.022829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addbe983-4fe1-4f74-8c16-f8e78ffdb04b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:51.024498 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addbe983-4fe1-4f74-8c16-f8e78ffdb04b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:51.040409 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 433/848] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:51.046515 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41421fa-d9d7-4177-9500-e3d35247a42d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:51.161349 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41421fa-d9d7-4177-9500-e3d35247a42d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:51.181432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27513882-0937-4882-9356-99d5e54acf96 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:51.181747 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27513882-0937-4882-9356-99d5e54acf96 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:51.181993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27513882-0937-4882-9356-99d5e54acf96 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:51.183675 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 434/849] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:14:51.189248 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ffa7b-c280-45fb-8055-b7c31e8caa35 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:51.189538 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ffa7b-c280-45fb-8055-b7c31e8caa35 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:51.189817 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ffa7b-c280-45fb-8055-b7c31e8caa35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:51.199739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7035450-029a-4a43-a3f1-34edd36f58e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:51.201729 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7035450-029a-4a43-a3f1-34edd36f58e7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:51.201729 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7035450-029a-4a43-a3f1-34edd36f58e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:51.201729 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7035450-029a-4a43-a3f1-34edd36f58e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:51.211062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7035450-029a-4a43-a3f1-34edd36f58e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:51.212654 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7035450-029a-4a43-a3f1-34edd36f58e7 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:51.213133 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-254ffa7b-c280-45fb-8055-b7c31e8caa35 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:51.214421 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-254ffa7b-c280-45fb-8055-b7c31e8caa35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:51.227873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-f7035450-029a-4a43-a3f1-34edd36f58e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:51.227873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:51.227873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:51.227873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:51.227873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:51.227873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:51.227873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:51.227873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:51.227873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:51.227873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:51.227873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:51.227873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:51.227873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:14:51.227873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:14:51.227873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:14:51.228816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:14:51.228816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:51.228816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:51.228816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:51.228816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:51.228816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:51.228816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:51.228816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:51.228816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:51.228816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:51.228816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:51.228816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:51.228816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:51.228816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:51.228816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:51.228816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:51.228816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:51.229710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:14:51.229710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:14:51.229710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:14:51.229710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:14:51.229710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:14:51.229710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:14:51.229710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:51.229710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:51.229710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 416/850] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:51.234866 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7035450-029a-4a43-a3f1-34edd36f58e7 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:51.247715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:51.288191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:51.304246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 435/851] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 10:14:51.305407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1fc9ac-49e7-4bb7-8d75-4c854174ca89 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:51.305690 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1fc9ac-49e7-4bb7-8d75-4c854174ca89 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:51.305967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1fc9ac-49e7-4bb7-8d75-4c854174ca89 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:51.311297 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 436/852] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:51.312435 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6d631-7796-4f60-a9ee-4456575dda0b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:51.312769 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6d631-7796-4f60-a9ee-4456575dda0b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:51.313046 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6d631-7796-4f60-a9ee-4456575dda0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:51.330737 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-38a6d631-7796-4f60-a9ee-4456575dda0b None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:51.332173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-38a6d631-7796-4f60-a9ee-4456575dda0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:51.358386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7035450-029a-4a43-a3f1-34edd36f58e7 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:51.407430 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 437/853] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:14:51.407430 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6d631-7796-4f60-a9ee-4456575dda0b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:51.408400 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4cd22-0c34-46d5-a711-f2f46613959f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:51.409152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4cd22-0c34-46d5-a711-f2f46613959f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:51.409443 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4cd22-0c34-46d5-a711-f2f46613959f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:51.409723 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f4cd22-0c34-46d5-a711-f2f46613959f None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:51.410523 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4cd22-0c34-46d5-a711-f2f46613959f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:51.410833 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4cd22-0c34-46d5-a711-f2f46613959f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:51.412531 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f4cd22-0c34-46d5-a711-f2f46613959f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:51.428703 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 417/854] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:14:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:51.428703 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4cd22-0c34-46d5-a711-f2f46613959f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:51.509248 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:51.524641 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a6d631-7796-4f60-a9ee-4456575dda0b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:51.544154 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f4cd22-0c34-46d5-a711-f2f46613959f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:51.554745 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:51.562120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec6a34-0704-42b0-b45e-f56904df4ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:51.562922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec6a34-0704-42b0-b45e-f56904df4ac0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:51.566163 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec6a34-0704-42b0-b45e-f56904df4ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:51.566163 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec6a34-0704-42b0-b45e-f56904df4ac0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:51.575508 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec6a34-0704-42b0-b45e-f56904df4ac0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:51.583981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-85ec6a34-0704-42b0-b45e-f56904df4ac0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:51.583981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:51.583981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:51.583981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:51.583981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:51.583981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:51.583981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:51.583981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:51.583981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:51.583981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:51.583981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:51.583981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:51.583981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:51.583981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:51.583981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:51.583981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:51.585398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:51.585398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:51.585398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:51.585398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:51.585398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:51.585398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:51.585398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:51.585398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:51.585398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:51.585398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:51.585398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:51.585398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:51.585398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:51.585398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:51.585398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:51.585398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:51.586530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:51.586530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:51.586530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:51.586530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:51.586530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:51.586530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:51.586530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:51.586530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:51.586530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:51.586530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:51.586530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:14:51.586530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:51.586530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 438/855] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jun 30 10:14:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:51.586530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec6a34-0704-42b0-b45e-f56904df4ac0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:51.596857 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff9cb0-e10d-419e-8dd8-4d1c4b327352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:51.597675 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff9cb0-e10d-419e-8dd8-4d1c4b327352 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:51.597958 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff9cb0-e10d-419e-8dd8-4d1c4b327352 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:51.598235 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff9cb0-e10d-419e-8dd8-4d1c4b327352 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:51.598892 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff9cb0-e10d-419e-8dd8-4d1c4b327352 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:51.600296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff9cb0-e10d-419e-8dd8-4d1c4b327352 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:51.612740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-70ff9cb0-e10d-419e-8dd8-4d1c4b327352 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 10:14:51.612740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:51.612740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 10:14:51.612740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 10:14:51.612740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 10:14:51.612740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 10:14:51.612740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 10:14:51.612740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:14:51.612740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 10:14:51.612740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:51.612740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:14:51.612740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:51.612740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:51.612740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:51.612740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:51.612740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:51.613676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:51.613676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:51.613676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:51.613676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:51.613676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:51.613676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:51.613676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:51.613676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 10:14:51.613676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 10:14:51.613676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 10:14:51.613676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 10:14:51.613676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:51.613676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:51.613676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:51.613676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:51.613676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:51.614503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:51.614503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:51.614503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:51.614503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:51.614503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:51.614503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:51.614503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:51.614503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:51.614503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:51.614503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:51.614503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:51.614503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 10:14:51.614503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:14:51.614503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 10:14:51.614503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:51.614503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 418/856] 10.222.0.47 () {56 vars in 1132 bytes} [Thu Jun 30 10:14:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:51.615374 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff9cb0-e10d-419e-8dd8-4d1c4b327352 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:51.700706 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec6a34-0704-42b0-b45e-f56904df4ac0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:51.728967 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff9cb0-e10d-419e-8dd8-4d1c4b327352 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:51.734878 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e1ea4a-4c03-4d1a-bd19-a0534a02ba6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:51.736305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1ea4a-4c03-4d1a-bd19-a0534a02ba6d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:51.736588 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1ea4a-4c03-4d1a-bd19-a0534a02ba6d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:51.736875 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e1ea4a-4c03-4d1a-bd19-a0534a02ba6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:51.737717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e1ea4a-4c03-4d1a-bd19-a0534a02ba6d None admin] RBAC: Adding query filter params (role.id=e63bda042e104a4a93fba15ffcfe1183, scope.project.id=50902e0539434c82b505dc5009aeaf85, user.id=07de926cb8594b098f79111dab8a9cc6) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:51.738030 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e1ea4a-4c03-4d1a-bd19-a0534a02ba6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:51.739585 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e1ea4a-4c03-4d1a-bd19-a0534a02ba6d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:51.750250 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 439/857] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:14:51 2022] GET /identity/v3/role_assignments?user.id=07de926cb8594b098f79111dab8a9cc6&scope.project.id=50902e0539434c82b505dc5009aeaf85&role.id=e63bda042e104a4a93fba15ffcfe1183 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:51.751544 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e1ea4a-4c03-4d1a-bd19-a0534a02ba6d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:51.774682 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:51.794505 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abedd368-17d9-4e22-949e-989179fff58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:51.795203 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abedd368-17d9-4e22-949e-989179fff58d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:51.795513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abedd368-17d9-4e22-949e-989179fff58d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:51.795789 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abedd368-17d9-4e22-949e-989179fff58d None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:51.796569 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abedd368-17d9-4e22-949e-989179fff58d None admin] RBAC: Adding query filter params (name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:51.796877 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abedd368-17d9-4e22-949e-989179fff58d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:51.798637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abedd368-17d9-4e22-949e-989179fff58d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:51.814799 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 419/858] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jun 30 10:14:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:51.815736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abedd368-17d9-4e22-949e-989179fff58d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:51.853871 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:51.880035 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e1ea4a-4c03-4d1a-bd19-a0534a02ba6d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:51.917194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d013ff6e-4c47-4bb2-8a03-9f0dfa3ec9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:51.918139 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013ff6e-4c47-4bb2-8a03-9f0dfa3ec9ce None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:51.918569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013ff6e-4c47-4bb2-8a03-9f0dfa3ec9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:51.918991 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d013ff6e-4c47-4bb2-8a03-9f0dfa3ec9ce None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:51.919912 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d013ff6e-4c47-4bb2-8a03-9f0dfa3ec9ce None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:51.920397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d013ff6e-4c47-4bb2-8a03-9f0dfa3ec9ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:51.921939 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d013ff6e-4c47-4bb2-8a03-9f0dfa3ec9ce None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:51.927430 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 440/859] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jun 30 10:14:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:51.932661 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d013ff6e-4c47-4bb2-8a03-9f0dfa3ec9ce None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:51.943350 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abedd368-17d9-4e22-949e-989179fff58d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:51.988658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2048bf-d661-4b8f-adc6-bcd19474d084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:51.989456 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2048bf-d661-4b8f-adc6-bcd19474d084 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:51.989734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2048bf-d661-4b8f-adc6-bcd19474d084 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:51.990005 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2048bf-d661-4b8f-adc6-bcd19474d084 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:52.004137 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2048bf-d661-4b8f-adc6-bcd19474d084 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:52.005591 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2048bf-d661-4b8f-adc6-bcd19474d084 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:52.020076 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:52.025261 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-db2048bf-d661-4b8f-adc6-bcd19474d084 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:52.025261 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:52.025261 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:52.025261 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:52.025261 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:52.025261 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:52.025261 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:52.025261 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:52.025261 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:52.025261 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:52.025261 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:52.025261 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:52.025261 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:52.025261 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:52.025261 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:52.026146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:52.026146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:52.026146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:52.026146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:52.026146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:52.026146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:52.026146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:52.026146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:52.026146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:52.026146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:52.026146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:52.026146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:52.026146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:52.026146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:52.026146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:52.026146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:52.026146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:52.027078 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:52.027078 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:52.027078 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:52.027078 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:52.027078 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 420/860] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:52.032899 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2048bf-d661-4b8f-adc6-bcd19474d084 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:52.043435 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d013ff6e-4c47-4bb2-8a03-9f0dfa3ec9ce None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:52.100126 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327907f7-a5a3-43dc-b5ec-fec4d7c8a6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:52.100126 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327907f7-a5a3-43dc-b5ec-fec4d7c8a6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:52.100126 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327907f7-a5a3-43dc-b5ec-fec4d7c8a6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:52.100126 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327907f7-a5a3-43dc-b5ec-fec4d7c8a6a6 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:52.100564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327907f7-a5a3-43dc-b5ec-fec4d7c8a6a6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:52.100564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327907f7-a5a3-43dc-b5ec-fec4d7c8a6a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:52.101624 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327907f7-a5a3-43dc-b5ec-fec4d7c8a6a6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:52.107307 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 441/861] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jun 30 10:14:51 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:14:52.147600 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:52.165322 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2048bf-d661-4b8f-adc6-bcd19474d084 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:52.207850 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb5d74-5300-449b-9ad4-57bd4f29eb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:52.208543 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb5d74-5300-449b-9ad4-57bd4f29eb86 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:52.208834 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb5d74-5300-449b-9ad4-57bd4f29eb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:52.209117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffb5d74-5300-449b-9ad4-57bd4f29eb86 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:52.209865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb5d74-5300-449b-9ad4-57bd4f29eb86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:52.210178 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb5d74-5300-449b-9ad4-57bd4f29eb86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:52.211627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffb5d74-5300-449b-9ad4-57bd4f29eb86 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:52.225134 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 421/862] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:14:52.230677 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327907f7-a5a3-43dc-b5ec-fec4d7c8a6a6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:52.341929 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327907f7-a5a3-43dc-b5ec-fec4d7c8a6a6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:52.378470 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a43f0-d927-4def-b9b1-6d5de23fd14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:52.379471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a43f0-d927-4def-b9b1-6d5de23fd14a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:52.379756 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a43f0-d927-4def-b9b1-6d5de23fd14a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:52.380028 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a43f0-d927-4def-b9b1-6d5de23fd14a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:52.388000 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a43f0-d927-4def-b9b1-6d5de23fd14a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:52.389503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a43f0-d927-4def-b9b1-6d5de23fd14a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:52.392324 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:52.405344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-994a43f0-d927-4def-b9b1-6d5de23fd14a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 10:14:52.405344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:52.405344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:52.405344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:52.405344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:52.405344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:52.405344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:52.405344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:52.405344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:52.405344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:52.405344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:52.405344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:52.405344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:14:52.405344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:14:52.405344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:14:52.405344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:14:52.410103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:52.410103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:52.410103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:14:52.410103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:52.410103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:14:52.410103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:52.410103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:52.410103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:52.410103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:52.410103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:52.410103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:52.410103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:52.410103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:52.410103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:52.410103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:52.410103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:52.410103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:52.410103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:52.411065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:52.411065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:52.411065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:14:52.411065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:14:52.411065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:14:52.411065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:14:52.411065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:14:52.411065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:14:52.411065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 10:14:52.411065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:52.411065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 442/863] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jun 30 10:14:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:52.413433 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb5d74-5300-449b-9ad4-57bd4f29eb86 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:52.531080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffb5d74-5300-449b-9ad4-57bd4f29eb86 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:52.566458 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121958f-93eb-4ee4-a021-2b8309b19cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:52.567540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121958f-93eb-4ee4-a021-2b8309b19cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:52.568030 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121958f-93eb-4ee4-a021-2b8309b19cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:52.568525 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d121958f-93eb-4ee4-a021-2b8309b19cf7 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:52.569569 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d121958f-93eb-4ee4-a021-2b8309b19cf7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:52.570099 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d121958f-93eb-4ee4-a021-2b8309b19cf7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:52.571885 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d121958f-93eb-4ee4-a021-2b8309b19cf7 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:52.590800 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 422/864] 10.222.0.47 () {56 vars in 1149 bytes} [Thu Jun 30 10:14:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:52.596612 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a43f0-d927-4def-b9b1-6d5de23fd14a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:52.701653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a43f0-d927-4def-b9b1-6d5de23fd14a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:52.739456 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a865f-bb60-4403-b8db-db3c464f99dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:52.740489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a865f-bb60-4403-b8db-db3c464f99dd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:52.740999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a865f-bb60-4403-b8db-db3c464f99dd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:52.741479 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3a865f-bb60-4403-b8db-db3c464f99dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:52.811190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a865f-bb60-4403-b8db-db3c464f99dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:52.812749 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3a865f-bb60-4403-b8db-db3c464f99dd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:52.823476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:52.914718 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 443/865] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:14:52 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 10:14:52.920159 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121958f-93eb-4ee4-a021-2b8309b19cf7 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:52.923721 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:53.020644 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d121958f-93eb-4ee4-a021-2b8309b19cf7 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:53.060098 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c183e-b5a4-46c1-a0ce-9f9f78ff2060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:53.060905 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c183e-b5a4-46c1-a0ce-9f9f78ff2060 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:53.061190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c183e-b5a4-46c1-a0ce-9f9f78ff2060 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:53.061468 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c183e-b5a4-46c1-a0ce-9f9f78ff2060 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:53.072618 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c183e-b5a4-46c1-a0ce-9f9f78ff2060 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:53.074076 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c183e-b5a4-46c1-a0ce-9f9f78ff2060 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:53.092823 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-a27c183e-b5a4-46c1-a0ce-9f9f78ff2060 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:53.092823 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:53.092823 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:53.092823 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:53.092823 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:53.092823 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:53.092823 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:53.092823 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:53.092823 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:53.092823 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:53.092823 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:53.092823 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:53.092823 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:53.092823 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:53.092823 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:53.093798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:53.093798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:53.093798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:53.093798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:53.093798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:53.093798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:53.093798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:53.093798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:53.093798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:53.093798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:53.093798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:53.093798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:53.093798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:53.093798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:53.093798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:53.093798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:53.093798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:53.095481 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:53.095481 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:53.095481 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:53.095481 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:53.095481 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 423/866] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:53.100391 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a865f-bb60-4403-b8db-db3c464f99dd None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:53.145813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd4228-1049-4096-964a-2452f8b9c603 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:53.146107 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd4228-1049-4096-964a-2452f8b9c603 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:53.146388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dd4228-1049-4096-964a-2452f8b9c603 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:53.148436 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 424/867] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:14:53.153801 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c45d0-449e-4fe8-aa60-354b61ceb8a8 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:53.154354 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c45d0-449e-4fe8-aa60-354b61ceb8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:53.154449 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9c45d0-449e-4fe8-aa60-354b61ceb8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:53.182873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-4d9c45d0-449e-4fe8-aa60-354b61ceb8a8 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:53.182873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-4d9c45d0-449e-4fe8-aa60-354b61ceb8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:53.206335 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:53.209320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3a865f-bb60-4403-b8db-db3c464f99dd None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:53.245165 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafac650-32cd-43b3-a1fd-68780298f26a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:53.245826 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafac650-32cd-43b3-a1fd-68780298f26a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:53.246105 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafac650-32cd-43b3-a1fd-68780298f26a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:53.246379 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafac650-32cd-43b3-a1fd-68780298f26a None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:53.247096 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafac650-32cd-43b3-a1fd-68780298f26a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:53.247447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafac650-32cd-43b3-a1fd-68780298f26a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:53.248821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafac650-32cd-43b3-a1fd-68780298f26a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:53.250305 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:53.265712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 444/868] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:53.266876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafac650-32cd-43b3-a1fd-68780298f26a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:53.279349 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 425/869] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:53 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 10:14:53.296656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c45d0-449e-4fe8-aa60-354b61ceb8a8 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:53.382982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafac650-32cd-43b3-a1fd-68780298f26a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:53.422229 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9c45d0-449e-4fe8-aa60-354b61ceb8a8 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:53.428670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd1b27-8338-4b6b-bfa7-d9fc93e3d879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:53.429772 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd1b27-8338-4b6b-bfa7-d9fc93e3d879 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:53.430706 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd1b27-8338-4b6b-bfa7-d9fc93e3d879 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:53.431187 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd1b27-8338-4b6b-bfa7-d9fc93e3d879 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:53.441477 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd1b27-8338-4b6b-bfa7-d9fc93e3d879 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:53.443923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd1b27-8338-4b6b-bfa7-d9fc93e3d879 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:53.460666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6fe84-9156-4127-a895-c6be653ffbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:53.461336 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6fe84-9156-4127-a895-c6be653ffbe2 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:53.461627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6fe84-9156-4127-a895-c6be653ffbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:53.461910 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6fe84-9156-4127-a895-c6be653ffbe2 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:53.462637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a6fe84-9156-4127-a895-c6be653ffbe2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:53.463157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-e6fd1b27-8338-4b6b-bfa7-d9fc93e3d879 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:53.463157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:53.463157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:53.463157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:53.463157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:53.463157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:53.463157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:53.463157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:53.463157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:53.463157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:53.463157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:53.463157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:53.463157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:14:53.463157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:14:53.463157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:14:53.464042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:14:53.464042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:53.464042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:53.464042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:53.464042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:53.464042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:53.464042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:53.464042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:53.464042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:53.464042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:53.464042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:53.464042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:53.464042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:53.464042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:53.464042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:53.464042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:53.464042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:53.464972 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:14:53.464972 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:14:53.464972 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:14:53.464972 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:14:53.464972 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:14:53.464972 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:14:53.464972 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:53.464972 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:53.465416 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a6fe84-9156-4127-a895-c6be653ffbe2 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:53.468160 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 445/870] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:14:53.474202 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd1b27-8338-4b6b-bfa7-d9fc93e3d879 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:53.497130 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:53.540951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 426/871] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jun 30 10:14:53 2022] POST /identity/v3/services => generated 251 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:14:53.595206 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd1b27-8338-4b6b-bfa7-d9fc93e3d879 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:53.644619 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a124042-f577-4bf0-b493-6eadd9bbd828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:53.645296 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a124042-f577-4bf0-b493-6eadd9bbd828 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:53.645609 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a124042-f577-4bf0-b493-6eadd9bbd828 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:53.645883 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a124042-f577-4bf0-b493-6eadd9bbd828 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:53.647043 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a124042-f577-4bf0-b493-6eadd9bbd828 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:53.647402 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a124042-f577-4bf0-b493-6eadd9bbd828 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:53.648908 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a124042-f577-4bf0-b493-6eadd9bbd828 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:53.663478 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 446/872] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:14:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:14:53.669584 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6fe84-9156-4127-a895-c6be653ffbe2 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:53.725293 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:53.784051 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6fe84-9156-4127-a895-c6be653ffbe2 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:53.820642 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e87ca2b-41d6-4e83-9a8c-eeeb4121a65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:53.821661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87ca2b-41d6-4e83-9a8c-eeeb4121a65e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:53.822179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87ca2b-41d6-4e83-9a8c-eeeb4121a65e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:53.822660 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e87ca2b-41d6-4e83-9a8c-eeeb4121a65e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:53.823730 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e87ca2b-41d6-4e83-9a8c-eeeb4121a65e None admin] RBAC: Adding query filter params (role.id=99d8cd9cd87e472290bf0de2710b5857, scope.project.id=50902e0539434c82b505dc5009aeaf85, user.id=66ad8defb9b6453c96f1a8a7343695ba) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:53.824250 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e87ca2b-41d6-4e83-9a8c-eeeb4121a65e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:53.825882 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e87ca2b-41d6-4e83-9a8c-eeeb4121a65e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:53.839455 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 427/873] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:14:53 2022] GET /identity/v3/role_assignments?user.id=66ad8defb9b6453c96f1a8a7343695ba&scope.project.id=50902e0539434c82b505dc5009aeaf85&role.id=99d8cd9cd87e472290bf0de2710b5857 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:54.077798 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d175de-dac1-42c2-a9a8-f04804c75c26 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:54.078138 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d175de-dac1-42c2-a9a8-f04804c75c26 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:54.078314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d175de-dac1-42c2-a9a8-f04804c75c26 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:54.080521 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 447/874] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:54.086727 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46c999-a5ae-4c9d-a5f3-c5ffcb6606bb None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:54.087299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46c999-a5ae-4c9d-a5f3-c5ffcb6606bb None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:54.087299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd46c999-a5ae-4c9d-a5f3-c5ffcb6606bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:54.108888 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-cd46c999-a5ae-4c9d-a5f3-c5ffcb6606bb None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:54.110119 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-cd46c999-a5ae-4c9d-a5f3-c5ffcb6606bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:54.181564 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 428/875] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:54 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:14:54.205021 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d175de-dac1-42c2-a9a8-f04804c75c26 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:54.288635 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:54.314409 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d175de-dac1-42c2-a9a8-f04804c75c26 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:54.318029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100e337-9765-4e3c-8dbc-e2200c1cb4bc None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:54.318029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100e337-9765-4e3c-8dbc-e2200c1cb4bc None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:54.318239 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100e337-9765-4e3c-8dbc-e2200c1cb4bc None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:54.321169 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 429/876] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:54.326387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e97c26-3b47-4103-a207-08a11d581078 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:54.326679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e97c26-3b47-4103-a207-08a11d581078 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:54.326957 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e97c26-3b47-4103-a207-08a11d581078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:54.345163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-12e97c26-3b47-4103-a207-08a11d581078 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:54.346377 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-12e97c26-3b47-4103-a207-08a11d581078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:54.356977 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacaf029-02ed-4f6d-b45c-165755e27e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:54.358472 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaf029-02ed-4f6d-b45c-165755e27e7e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:54.359112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaf029-02ed-4f6d-b45c-165755e27e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:54.359731 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacaf029-02ed-4f6d-b45c-165755e27e7e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:54.361160 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacaf029-02ed-4f6d-b45c-165755e27e7e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:54.363626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacaf029-02ed-4f6d-b45c-165755e27e7e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:54.379908 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:54.382559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-bacaf029-02ed-4f6d-b45c-165755e27e7e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 10:14:54.382559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:54.382559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 10:14:54.382559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 10:14:54.382559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 10:14:54.382559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 10:14:54.382559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 10:14:54.382559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:14:54.382559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 10:14:54.382559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:54.382559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:14:54.382559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:54.382559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:54.382559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:54.382559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:54.382559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:54.382559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:54.384002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:54.384002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:54.384002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:54.384002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:54.384002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:54.384002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:54.384002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 10:14:54.384002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 10:14:54.384002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 10:14:54.384002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 10:14:54.384002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:54.384002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:54.384002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:54.384002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:54.384002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:54.384002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:54.384841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:54.384841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:54.384841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:54.384841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:54.384841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:54.384841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:54.384841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:54.384841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:54.384841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:54.384841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:54.384841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 10:14:54.384841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:14:54.384841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 10:14:54.384841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:54.385579 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 448/877] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jun 30 10:14:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:54.392168 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacaf029-02ed-4f6d-b45c-165755e27e7e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:54.443354 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 430/878] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:54 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 10:14:54.458085 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e97c26-3b47-4103-a207-08a11d581078 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:54.521425 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacaf029-02ed-4f6d-b45c-165755e27e7e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:54.565209 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f8179-d92c-4cc9-a193-57537a046632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:54.565935 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f8179-d92c-4cc9-a193-57537a046632 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:54.566225 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f8179-d92c-4cc9-a193-57537a046632 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:54.566502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17f8179-d92c-4cc9-a193-57537a046632 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:54.567213 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f8179-d92c-4cc9-a193-57537a046632 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:54.567542 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f8179-d92c-4cc9-a193-57537a046632 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:54.568924 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17f8179-d92c-4cc9-a193-57537a046632 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:54.576527 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 449/879] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jun 30 10:14:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:54.582837 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f8179-d92c-4cc9-a193-57537a046632 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:54.586987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e97c26-3b47-4103-a207-08a11d581078 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:54.587970 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:54.627079 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ce901-55dd-4d06-99c5-cdcafb2dc0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:54.628183 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ce901-55dd-4d06-99c5-cdcafb2dc0fd None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:54.628471 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ce901-55dd-4d06-99c5-cdcafb2dc0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:54.628747 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ce901-55dd-4d06-99c5-cdcafb2dc0fd None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:54.629594 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ce901-55dd-4d06-99c5-cdcafb2dc0fd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:54.630991 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ce901-55dd-4d06-99c5-cdcafb2dc0fd None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:54.703425 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17f8179-d92c-4cc9-a193-57537a046632 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:54.735314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 431/880] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jun 30 10:14:54 2022] POST /identity/v3/services => generated 269 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:14:54.759975 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afac8bc2-2e5d-4b80-b0ee-def8d526e690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:54.760662 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac8bc2-2e5d-4b80-b0ee-def8d526e690 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:54.760946 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac8bc2-2e5d-4b80-b0ee-def8d526e690 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:54.761281 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afac8bc2-2e5d-4b80-b0ee-def8d526e690 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:54.762068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac8bc2-2e5d-4b80-b0ee-def8d526e690 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:54.762420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac8bc2-2e5d-4b80-b0ee-def8d526e690 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:54.764205 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afac8bc2-2e5d-4b80-b0ee-def8d526e690 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:54.770348 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 450/881] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jun 30 10:14:54 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:55.671462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d7d4c-ad94-4d17-8d02-dcaebbc4ec8d None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:55.672003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d7d4c-ad94-4d17-8d02-dcaebbc4ec8d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:55.672003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d7d4c-ad94-4d17-8d02-dcaebbc4ec8d None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:55.674334 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 432/882] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:55.679709 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a57ea8-7398-48e2-a28d-981b57946467 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:55.680584 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a57ea8-7398-48e2-a28d-981b57946467 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:55.681072 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a57ea8-7398-48e2-a28d-981b57946467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:55.697917 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b4a57ea8-7398-48e2-a28d-981b57946467 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:55.699418 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b4a57ea8-7398-48e2-a28d-981b57946467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:55.758666 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:55.770909 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 451/883] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:55 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:14:55.794406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d7d4c-ad94-4d17-8d02-dcaebbc4ec8d None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:55.824183 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:55.909718 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d7d4c-ad94-4d17-8d02-dcaebbc4ec8d None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:55.948805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d780a-7043-4bfd-9645-bad075254e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:55.949608 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d780a-7043-4bfd-9645-bad075254e9c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:55.949893 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d780a-7043-4bfd-9645-bad075254e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:55.950168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d780a-7043-4bfd-9645-bad075254e9c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:55.950829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d780a-7043-4bfd-9645-bad075254e9c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:55.952239 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d780a-7043-4bfd-9645-bad075254e9c None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:55.964633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-173d780a-7043-4bfd-9645-bad075254e9c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 10:14:55.964633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:55.964633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 10:14:55.964633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 10:14:55.964633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 10:14:55.964633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 10:14:55.964633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 10:14:55.964633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:14:55.964633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 10:14:55.964633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:55.964633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:14:55.964633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:55.964633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:55.964633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:55.964633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:55.964633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:55.964633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:55.965637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:55.965637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:55.965637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:55.965637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:55.965637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:55.965637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:55.965637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 10:14:55.965637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 10:14:55.965637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 10:14:55.965637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 10:14:55.965637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:55.965637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:55.965637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:55.965637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:55.965637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:55.965637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:55.966632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:55.966632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:55.966632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:55.966632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:55.966632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:55.966632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:55.966632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:55.966632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:55.966632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:55.966632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:55.966632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 10:14:55.966632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:14:55.966632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 10:14:55.966632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:55.966632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 433/884] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jun 30 10:14:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:55.974597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a57ea8-7398-48e2-a28d-981b57946467 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:56.035598 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1eed0-dbe5-4cc5-bb96-c8d0d23fd729 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:56.036415 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1eed0-dbe5-4cc5-bb96-c8d0d23fd729 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:56.036908 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be1eed0-dbe5-4cc5-bb96-c8d0d23fd729 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:56.042432 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 434/885] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:14:56.044205 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299801b-2938-43e4-86af-2b3523b9401f None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:56.044721 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299801b-2938-43e4-86af-2b3523b9401f None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:56.045271 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5299801b-2938-43e4-86af-2b3523b9401f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:56.065248 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-5299801b-2938-43e4-86af-2b3523b9401f None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:56.066868 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-5299801b-2938-43e4-86af-2b3523b9401f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:56.095896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a57ea8-7398-48e2-a28d-981b57946467 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:56.134956 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed7123-a854-4bbd-9199-4426fdf726ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:56.136042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed7123-a854-4bbd-9199-4426fdf726ba None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:56.136561 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed7123-a854-4bbd-9199-4426fdf726ba None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:56.137585 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed7123-a854-4bbd-9199-4426fdf726ba None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:56.139260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed7123-a854-4bbd-9199-4426fdf726ba None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:56.147627 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed7123-a854-4bbd-9199-4426fdf726ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:56.147627 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed7123-a854-4bbd-9199-4426fdf726ba None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:56.151530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 452/886] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jun 30 10:14:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:56.156550 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed7123-a854-4bbd-9199-4426fdf726ba None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:56.167286 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 435/887] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:56 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 10:14:56.185653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299801b-2938-43e4-86af-2b3523b9401f None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:56.216739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:56.225374 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:56.279629 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed7123-a854-4bbd-9199-4426fdf726ba None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:56.302585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5299801b-2938-43e4-86af-2b3523b9401f None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:56.322659 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31173d3-cd50-45b3-b9d3-ab20782765b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:56.323353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31173d3-cd50-45b3-b9d3-ab20782765b5 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:56.323644 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31173d3-cd50-45b3-b9d3-ab20782765b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:56.323922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31173d3-cd50-45b3-b9d3-ab20782765b5 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:56.324640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31173d3-cd50-45b3-b9d3-ab20782765b5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:56.324948 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31173d3-cd50-45b3-b9d3-ab20782765b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:56.326962 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31173d3-cd50-45b3-b9d3-ab20782765b5 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:56.335393 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 453/888] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jun 30 10:14:56 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:14:56.339495 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31173d3-cd50-45b3-b9d3-ab20782765b5 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:56.347912 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9bdf58-7c67-4b9b-bf1a-c833ca957de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:56.348957 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bdf58-7c67-4b9b-bf1a-c833ca957de1 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:56.349399 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bdf58-7c67-4b9b-bf1a-c833ca957de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:56.349916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9bdf58-7c67-4b9b-bf1a-c833ca957de1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:56.364030 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9bdf58-7c67-4b9b-bf1a-c833ca957de1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:14:56.375296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-8f9bdf58-7c67-4b9b-bf1a-c833ca957de1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 10:14:56.375296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:56.375296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:56.375296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:56.375296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:56.375296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:56.375296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:56.375296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:56.375296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:56.375296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:56.375296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:56.375296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:56.375296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:14:56.375296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:14:56.375296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:14:56.375296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:14:56.376624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:14:56.376624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:14:56.376624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:14:56.376624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:14:56.376624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:56.376624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:56.376624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:56.376624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:56.376624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:56.376624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:56.376624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:56.376624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:56.376624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:56.376624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:56.376624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:56.376624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:56.377731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:56.377731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:56.377731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:56.377731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:56.377731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:14:56.377731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:14:56.377731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:14:56.377731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:14:56.377731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:14:56.377731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:14:56.377731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 10:14:56.377731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:56.380122 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 436/889] 10.222.0.47 () {56 vars in 1120 bytes} [Thu Jun 30 10:14:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:14:56.386436 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9bdf58-7c67-4b9b-bf1a-c833ca957de1 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:56.470620 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31173d3-cd50-45b3-b9d3-ab20782765b5 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:56.479968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:56.501458 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9bdf58-7c67-4b9b-bf1a-c833ca957de1 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:56.504761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:56.520881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52ad4a-292b-4835-a8eb-2c54831d038b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:56.521577 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52ad4a-292b-4835-a8eb-2c54831d038b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:56.522334 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52ad4a-292b-4835-a8eb-2c54831d038b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:56.522635 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52ad4a-292b-4835-a8eb-2c54831d038b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:56.523477 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e52ad4a-292b-4835-a8eb-2c54831d038b None admin] RBAC: Adding query filter params (interface=public, service_id=2e678548bf144e298d309f111c81f26d, region_id=RegionOne) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:56.523795 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e52ad4a-292b-4835-a8eb-2c54831d038b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:56.525657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e52ad4a-292b-4835-a8eb-2c54831d038b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:56.535340 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 454/890] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Jun 30 10:14:56 2022] GET /identity/v3/endpoints?service_id=2e678548bf144e298d309f111c81f26d&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:56.538466 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52ad4a-292b-4835-a8eb-2c54831d038b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:56.550446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d301d-6c63-4c26-b20b-dc1bf4d308a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:56.551134 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d301d-6c63-4c26-b20b-dc1bf4d308a0 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:56.551456 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d301d-6c63-4c26-b20b-dc1bf4d308a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:56.551736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d301d-6c63-4c26-b20b-dc1bf4d308a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:56.552871 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d301d-6c63-4c26-b20b-dc1bf4d308a0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:56.552871 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d301d-6c63-4c26-b20b-dc1bf4d308a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:56.554167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d301d-6c63-4c26-b20b-dc1bf4d308a0 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:56.569631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 437/891] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Jun 30 10:14:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:56.571023 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d301d-6c63-4c26-b20b-dc1bf4d308a0 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:56.671249 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52ad4a-292b-4835-a8eb-2c54831d038b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:56.681930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d301d-6c63-4c26-b20b-dc1bf4d308a0 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:56.713905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f0105-606e-4d64-9bb1-860a62282add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:56.714717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f0105-606e-4d64-9bb1-860a62282add None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:56.715329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f0105-606e-4d64-9bb1-860a62282add None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:56.715618 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f0105-606e-4d64-9bb1-860a62282add None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:56.716583 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f0105-606e-4d64-9bb1-860a62282add None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:56.718152 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f0105-606e-4d64-9bb1-860a62282add None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:56.721415 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a2d6c3-8367-40a3-83a0-53e896b05e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:56.722510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2d6c3-8367-40a3-83a0-53e896b05e6b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:56.723056 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2d6c3-8367-40a3-83a0-53e896b05e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:56.723583 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a2d6c3-8367-40a3-83a0-53e896b05e6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:56.727353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 455/892] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jun 30 10:14:56 2022] GET /identity/v3/services => generated 1268 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:14:56.742682 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a2d6c3-8367-40a3-83a0-53e896b05e6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:56.744599 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a2d6c3-8367-40a3-83a0-53e896b05e6b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:56.748168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:56.765483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-b9a2d6c3-8367-40a3-83a0-53e896b05e6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:56.765483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:56.765483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:56.765483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:56.765483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:56.765483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:56.765483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:56.765483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:56.765483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:56.765483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:56.765483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:56.765483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:56.765483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:56.765483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:56.765483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:56.768820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:56.768820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:56.768820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:56.768820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:56.768820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:56.768820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:56.768820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:56.768820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:56.768820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:56.768820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:56.768820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:56.768820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:56.768820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:56.768820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:56.768820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:56.768820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:56.768820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:56.769732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:56.769732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:56.769732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:56.769732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:56.772760 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 438/893] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:56.774295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f0105-606e-4d64-9bb1-860a62282add None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:56.834762 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:56.887725 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50603e-3059-4293-b4b2-b0f06228cd24 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:56.888408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50603e-3059-4293-b4b2-b0f06228cd24 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:56.888904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50603e-3059-4293-b4b2-b0f06228cd24 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:56.892740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 439/894] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:14:56.897917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5f988-5f8b-42a4-9f7c-2e3b28964884 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:56.898091 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f0105-606e-4d64-9bb1-860a62282add None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:56.898640 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5f988-5f8b-42a4-9f7c-2e3b28964884 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:56.899137 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f5f988-5f8b-42a4-9f7c-2e3b28964884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:56.917673 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-25f5f988-5f8b-42a4-9f7c-2e3b28964884 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:56.919226 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-25f5f988-5f8b-42a4-9f7c-2e3b28964884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:56.935819 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17de57f1-78b5-406e-9e89-4af2b6ded94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:56.936571 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de57f1-78b5-406e-9e89-4af2b6ded94f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:56.936931 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de57f1-78b5-406e-9e89-4af2b6ded94f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:56.937697 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17de57f1-78b5-406e-9e89-4af2b6ded94f None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:56.938461 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17de57f1-78b5-406e-9e89-4af2b6ded94f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:56.939008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17de57f1-78b5-406e-9e89-4af2b6ded94f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:56.940497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17de57f1-78b5-406e-9e89-4af2b6ded94f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:56.954754 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 456/895] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:56.955936 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855b5e0-3c3d-42a4-ac93-0d395531ba56 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:56.956229 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855b5e0-3c3d-42a4-ac93-0d395531ba56 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:56.956506 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a855b5e0-3c3d-42a4-ac93-0d395531ba56 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:56.957894 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 457/896] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:56.962600 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855b5e0-3c3d-42a4-ac93-0d395531ba56 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:57.002373 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 440/897] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:56 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 10:14:57.008093 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2108cbd-9181-46f9-9a83-b50fa97cc71c None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:57.009059 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2108cbd-9181-46f9-9a83-b50fa97cc71c None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:57.009550 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2108cbd-9181-46f9-9a83-b50fa97cc71c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:57.031317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-c2108cbd-9181-46f9-9a83-b50fa97cc71c None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:57.032678 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-c2108cbd-9181-46f9-9a83-b50fa97cc71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:57.070641 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a855b5e0-3c3d-42a4-ac93-0d395531ba56 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:57.083865 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:57.103109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 441/898] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:57 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:14:57.103715 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2108cbd-9181-46f9-9a83-b50fa97cc71c None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:57.118348 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a871e-95b1-43d5-9744-8a5b8650aa8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:57.120183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a871e-95b1-43d5-9744-8a5b8650aa8f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:57.120668 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a871e-95b1-43d5-9744-8a5b8650aa8f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:57.121398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a871e-95b1-43d5-9744-8a5b8650aa8f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:57.131881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a871e-95b1-43d5-9744-8a5b8650aa8f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:57.133346 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a871e-95b1-43d5-9744-8a5b8650aa8f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:57.160397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-a92a871e-95b1-43d5-9744-8a5b8650aa8f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 10:14:57.160397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:57.160397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:57.160397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:57.160397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:57.160397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:57.160397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:57.160397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:57.160397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:57.160397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:57.160397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:57.160397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:57.160397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:14:57.160397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:14:57.160397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:14:57.160397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:14:57.161420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:57.161420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:57.161420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:14:57.161420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:57.161420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:14:57.161420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:14:57.161420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:57.161420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:57.161420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:57.161420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:57.161420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:57.161420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:57.161420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:57.161420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:57.161420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:57.161420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:57.161420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:57.161420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:57.162382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:57.162382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:57.162382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:14:57.162382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:14:57.162382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:14:57.162382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:14:57.162382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:14:57.162382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:14:57.162382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 10:14:57.162382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:57.162382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 458/899] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jun 30 10:14:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:14:57.163009 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a871e-95b1-43d5-9744-8a5b8650aa8f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:57.184585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:57.241216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2108cbd-9181-46f9-9a83-b50fa97cc71c None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:57.277886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031023fc-d034-44e9-bfc9-480ea9f465f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:57.278376 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a871e-95b1-43d5-9744-8a5b8650aa8f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:57.278729 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031023fc-d034-44e9-bfc9-480ea9f465f9 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:57.279083 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031023fc-d034-44e9-bfc9-480ea9f465f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:57.279475 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031023fc-d034-44e9-bfc9-480ea9f465f9 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:57.280283 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031023fc-d034-44e9-bfc9-480ea9f465f9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:57.281756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031023fc-d034-44e9-bfc9-480ea9f465f9 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:57.313100 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c2f98-895b-4044-8730-c49a88e3b445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:57.314407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c2f98-895b-4044-8730-c49a88e3b445 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:57.314688 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c2f98-895b-4044-8730-c49a88e3b445 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:57.314963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557c2f98-895b-4044-8730-c49a88e3b445 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:57.315646 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c2f98-895b-4044-8730-c49a88e3b445 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:57.317022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557c2f98-895b-4044-8730-c49a88e3b445 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:57.329468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-557c2f98-895b-4044-8730-c49a88e3b445 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 10:14:57.329468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:57.329468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 10:14:57.329468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 10:14:57.329468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 10:14:57.329468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 10:14:57.329468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 10:14:57.329468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:14:57.329468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 10:14:57.329468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:57.329468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:14:57.329468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:57.329468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:57.329468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:57.329468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:57.329468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:57.330496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:57.330496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:57.330496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:57.330496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:57.330496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:57.330496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:57.330496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:57.330496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 10:14:57.330496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 10:14:57.330496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 10:14:57.330496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 10:14:57.330496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:57.330496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:57.330496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:57.330496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:57.330496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:57.331760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:57.331760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:57.331760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:57.331760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:57.331760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:57.331760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:57.331760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:57.331760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:57.331760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:57.331760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:57.331760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:57.331760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 10:14:57.331760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:14:57.331760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 10:14:57.331760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:57.331760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 459/900] 10.222.0.47 () {56 vars in 1132 bytes} [Thu Jun 30 10:14:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:57.332633 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c2f98-895b-4044-8730-c49a88e3b445 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:57.367318 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:57.387314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 442/901] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jun 30 10:14:57 2022] POST /identity/v3/services => generated 241 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:14:57.387314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031023fc-d034-44e9-bfc9-480ea9f465f9 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:57.469934 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557c2f98-895b-4044-8730-c49a88e3b445 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:57.532188 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38126206-1af4-48e7-9103-997869ec1b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:57.532872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38126206-1af4-48e7-9103-997869ec1b28 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:57.533166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38126206-1af4-48e7-9103-997869ec1b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:57.533445 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38126206-1af4-48e7-9103-997869ec1b28 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:57.534306 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031023fc-d034-44e9-bfc9-480ea9f465f9 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:57.534418 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38126206-1af4-48e7-9103-997869ec1b28 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:57.534677 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38126206-1af4-48e7-9103-997869ec1b28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:57.536557 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38126206-1af4-48e7-9103-997869ec1b28 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:57.557958 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:57.566434 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 460/902] 10.222.0.47 () {56 vars in 1149 bytes} [Thu Jun 30 10:14:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:57.575666 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38126206-1af4-48e7-9103-997869ec1b28 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:57.580093 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4adbf-e29b-42a5-955d-0a95159ff5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:57.580782 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4adbf-e29b-42a5-955d-0a95159ff5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:57.581915 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4adbf-e29b-42a5-955d-0a95159ff5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:57.582213 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d4adbf-e29b-42a5-955d-0a95159ff5e9 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:57.583091 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d4adbf-e29b-42a5-955d-0a95159ff5e9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:57.583454 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d4adbf-e29b-42a5-955d-0a95159ff5e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:57.584843 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d4adbf-e29b-42a5-955d-0a95159ff5e9 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:57.590934 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 443/903] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jun 30 10:14:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:57.598209 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4adbf-e29b-42a5-955d-0a95159ff5e9 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:57.684431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:57.699794 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38126206-1af4-48e7-9103-997869ec1b28 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:57.728023 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d4adbf-e29b-42a5-955d-0a95159ff5e9 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:57.746514 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f904f-304d-4fc4-8356-362a1b9b10f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:57.747580 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f904f-304d-4fc4-8356-362a1b9b10f2 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:57.747879 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f904f-304d-4fc4-8356-362a1b9b10f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:57.748163 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f904f-304d-4fc4-8356-362a1b9b10f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:57.765281 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269af329-3e81-49a5-a6a5-b263b70a98f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:57.765952 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269af329-3e81-49a5-a6a5-b263b70a98f9 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:57.766279 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269af329-3e81-49a5-a6a5-b263b70a98f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:57.766560 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269af329-3e81-49a5-a6a5-b263b70a98f9 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:57.767411 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269af329-3e81-49a5-a6a5-b263b70a98f9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:57.767723 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269af329-3e81-49a5-a6a5-b263b70a98f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:57.769260 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269af329-3e81-49a5-a6a5-b263b70a98f9 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:57.779358 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 444/904] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jun 30 10:14:57 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:57.781045 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269af329-3e81-49a5-a6a5-b263b70a98f9 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:57.834513 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f904f-304d-4fc4-8356-362a1b9b10f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:57.836766 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f904f-304d-4fc4-8356-362a1b9b10f2 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:57.838403 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:57.909843 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269af329-3e81-49a5-a6a5-b263b70a98f9 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:57.946604 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:57.948841 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0d042-1d7d-4561-bfa1-594cb9af4501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:57.949507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0d042-1d7d-4561-bfa1-594cb9af4501 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:57.949788 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0d042-1d7d-4561-bfa1-594cb9af4501 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:57.950061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c0d042-1d7d-4561-bfa1-594cb9af4501 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:57.950851 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c0d042-1d7d-4561-bfa1-594cb9af4501 None admin] RBAC: Adding query filter params (interface=public, service_id=a84d2350a83c4d7c88583914b69c2961, region_id=RegionOne) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:57.951156 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c0d042-1d7d-4561-bfa1-594cb9af4501 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:57.952649 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c0d042-1d7d-4561-bfa1-594cb9af4501 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:57.964058 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 445/905] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Jun 30 10:14:57 2022] GET /identity/v3/endpoints?service_id=a84d2350a83c4d7c88583914b69c2961&interface=public®ion_id=RegionOne => generated 194 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:14:57.965025 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0d042-1d7d-4561-bfa1-594cb9af4501 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:57.965202 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 461/906] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:14:57 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 10:14:57.966030 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f904f-304d-4fc4-8356-362a1b9b10f2 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:58.074247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c0d042-1d7d-4561-bfa1-594cb9af4501 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:58.084339 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:58.097349 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f904f-304d-4fc4-8356-362a1b9b10f2 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:58.111650 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e63ab4-04e8-4a75-9adb-bfa739061ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:58.112323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e63ab4-04e8-4a75-9adb-bfa739061ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:58.112611 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e63ab4-04e8-4a75-9adb-bfa739061ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:58.113871 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e63ab4-04e8-4a75-9adb-bfa739061ab0 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:58.114661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e63ab4-04e8-4a75-9adb-bfa739061ab0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:58.116073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e63ab4-04e8-4a75-9adb-bfa739061ab0 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:58.122768 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 446/907] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jun 30 10:14:57 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:14:58.145262 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767d53d-7e60-4f65-b266-5b199e7e1634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:58.146149 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767d53d-7e60-4f65-b266-5b199e7e1634 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:58.146436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767d53d-7e60-4f65-b266-5b199e7e1634 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:58.146969 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767d53d-7e60-4f65-b266-5b199e7e1634 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:58.160443 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767d53d-7e60-4f65-b266-5b199e7e1634 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:58.161935 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767d53d-7e60-4f65-b266-5b199e7e1634 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:58.181821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-c767d53d-7e60-4f65-b266-5b199e7e1634 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:58.181821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:58.181821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:58.181821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:58.181821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:58.181821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:58.181821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:58.181821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:58.181821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:58.181821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:58.181821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:58.181821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:58.181821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:14:58.181821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:14:58.181821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:14:58.182869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:14:58.182869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:58.182869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:58.182869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:58.182869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:58.182869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:58.182869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:58.182869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:58.182869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:58.182869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:58.182869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:58.182869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:58.182869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:58.182869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:58.182869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:58.182869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:58.182869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:58.183789 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:14:58.183789 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:14:58.183789 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:14:58.183789 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:58.183789 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 462/908] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:14:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:58.190138 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e63ab4-04e8-4a75-9adb-bfa739061ab0 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:58.310000 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e63ab4-04e8-4a75-9adb-bfa739061ab0 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:58.353008 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00f563-53ef-4595-b6e7-854d7d52810a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:58.353685 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00f563-53ef-4595-b6e7-854d7d52810a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:58.353971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00f563-53ef-4595-b6e7-854d7d52810a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:58.354251 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b00f563-53ef-4595-b6e7-854d7d52810a None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:58.354987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00f563-53ef-4595-b6e7-854d7d52810a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:58.355330 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00f563-53ef-4595-b6e7-854d7d52810a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:58.356713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b00f563-53ef-4595-b6e7-854d7d52810a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:58.370089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 447/909] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:58.375204 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767d53d-7e60-4f65-b266-5b199e7e1634 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:58.418606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:58.478081 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767d53d-7e60-4f65-b266-5b199e7e1634 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:58.520029 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad31d8-a67b-43a5-abc3-82d86426585c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:58.520837 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad31d8-a67b-43a5-abc3-82d86426585c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:58.521118 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad31d8-a67b-43a5-abc3-82d86426585c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:58.521393 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad31d8-a67b-43a5-abc3-82d86426585c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:58.528685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad31d8-a67b-43a5-abc3-82d86426585c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:58.530141 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad31d8-a67b-43a5-abc3-82d86426585c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:58.544245 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-0dad31d8-a67b-43a5-abc3-82d86426585c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:58.544245 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:58.544245 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:58.544245 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:58.544245 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:58.544245 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:58.544245 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:58.544245 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:58.544245 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:58.544245 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:58.544245 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:58.544245 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:58.544245 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:14:58.544245 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:14:58.544245 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:14:58.546093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:14:58.546093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:58.546093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:58.546093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:58.546093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:58.546093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:58.546093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:58.546093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:58.546093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:58.546093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:58.546093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:58.546093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:58.546093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:58.546093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:58.546093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:58.546093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:58.546093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:58.547631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:14:58.547631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:14:58.547631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:14:58.547631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:14:58.547631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:14:58.547631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:14:58.547631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:14:58.547631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:58.547631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 463/910] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:14:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:14:58.551499 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00f563-53ef-4595-b6e7-854d7d52810a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:58.585368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:58.658161 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b00f563-53ef-4595-b6e7-854d7d52810a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:58.696452 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b64a8-9170-4bd9-913c-7a933506c326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:58.697127 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b64a8-9170-4bd9-913c-7a933506c326 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:58.697413 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b64a8-9170-4bd9-913c-7a933506c326 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:58.697700 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b64a8-9170-4bd9-913c-7a933506c326 None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:58.698492 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b64a8-9170-4bd9-913c-7a933506c326 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:58.698804 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b64a8-9170-4bd9-913c-7a933506c326 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:58.700280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b64a8-9170-4bd9-913c-7a933506c326 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:58.713497 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 448/911] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:14:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:58.719011 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad31d8-a67b-43a5-abc3-82d86426585c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:58.820999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:58.829589 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad31d8-a67b-43a5-abc3-82d86426585c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:58.864450 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a4c34-b95d-4212-9d3b-13e77c03d1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:58.865030 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a4c34-b95d-4212-9d3b-13e77c03d1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:58.865314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a4c34-b95d-4212-9d3b-13e77c03d1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:58.865589 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21a4c34-b95d-4212-9d3b-13e77c03d1a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:58.866433 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a4c34-b95d-4212-9d3b-13e77c03d1a8 None admin] RBAC: Adding query filter params (role.id=0b2dccc298c3428f91cea20447956c6a, scope.project.id=50902e0539434c82b505dc5009aeaf85, user.id=66ad8defb9b6453c96f1a8a7343695ba) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:58.866749 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a4c34-b95d-4212-9d3b-13e77c03d1a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:58.868218 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21a4c34-b95d-4212-9d3b-13e77c03d1a8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:58.870898 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a19fc6-e3ed-4f37-a707-9ed604047f30 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:58.871395 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a19fc6-e3ed-4f37-a707-9ed604047f30 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:58.871475 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a19fc6-e3ed-4f37-a707-9ed604047f30 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:58.872972 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 449/912] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:58.878742 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 464/913] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:14:58 2022] GET /identity/v3/role_assignments?user.id=66ad8defb9b6453c96f1a8a7343695ba&scope.project.id=50902e0539434c82b505dc5009aeaf85&role.id=0b2dccc298c3428f91cea20447956c6a => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:58.879047 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b5e0d-582c-40e6-9baf-b8a84e314b1c None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:58.879598 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b5e0d-582c-40e6-9baf-b8a84e314b1c None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:58.879932 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81b5e0d-582c-40e6-9baf-b8a84e314b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:58.901997 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-d81b5e0d-582c-40e6-9baf-b8a84e314b1c None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:58.903374 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d81b5e0d-582c-40e6-9baf-b8a84e314b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:58.991522 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 450/914] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 10:14:59.015466 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a4c34-b95d-4212-9d3b-13e77c03d1a8 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:59.123031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21a4c34-b95d-4212-9d3b-13e77c03d1a8 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:59.159572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282dce66-3485-4ec7-82ce-43d9c88280a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:59.160387 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282dce66-3485-4ec7-82ce-43d9c88280a4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:59.160675 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282dce66-3485-4ec7-82ce-43d9c88280a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:59.160956 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282dce66-3485-4ec7-82ce-43d9c88280a4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:59.161618 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282dce66-3485-4ec7-82ce-43d9c88280a4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:59.163010 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282dce66-3485-4ec7-82ce-43d9c88280a4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:59.175502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-282dce66-3485-4ec7-82ce-43d9c88280a4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 10:14:59.175502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:59.175502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 10:14:59.175502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 10:14:59.175502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 10:14:59.175502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 10:14:59.175502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 10:14:59.175502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:14:59.175502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 10:14:59.175502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:59.175502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:14:59.175502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:59.175502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:59.175502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:59.175502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:59.175502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:59.175502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:59.176526 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:59.176526 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:59.176526 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:59.176526 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:59.176526 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:59.176526 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:59.176526 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 10:14:59.176526 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 10:14:59.176526 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 10:14:59.176526 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 10:14:59.176526 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:59.176526 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:59.176526 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:59.176526 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:59.176526 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:59.176526 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:59.177372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:59.177372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:59.177372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:59.177372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:59.177372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:59.177372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:59.177372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:59.177372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:59.177372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:59.177372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:59.177372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 10:14:59.177372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:14:59.177372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 30 10:14:59.177372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:14:59.181865 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 465/915] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jun 30 10:14:59 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:14:59.184372 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b5e0d-582c-40e6-9baf-b8a84e314b1c None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:59.210215 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:59.288413 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81b5e0d-582c-40e6-9baf-b8a84e314b1c None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:59.326491 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd22d9b-6e6b-4bc1-9363-d4ee09cdb467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:59.327170 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd22d9b-6e6b-4bc1-9363-d4ee09cdb467 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:59.327491 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd22d9b-6e6b-4bc1-9363-d4ee09cdb467 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:59.327766 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd22d9b-6e6b-4bc1-9363-d4ee09cdb467 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:59.329098 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd22d9b-6e6b-4bc1-9363-d4ee09cdb467 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:59.329410 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd22d9b-6e6b-4bc1-9363-d4ee09cdb467 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:59.330882 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd22d9b-6e6b-4bc1-9363-d4ee09cdb467 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:59.336310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 451/916] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jun 30 10:14:59 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:59.342035 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282dce66-3485-4ec7-82ce-43d9c88280a4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:59.372972 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:59.451229 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282dce66-3485-4ec7-82ce-43d9c88280a4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:59.488229 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23480248-0ac0-48ea-a179-3e50d3e1729b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:59.489305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23480248-0ac0-48ea-a179-3e50d3e1729b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:59.489795 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23480248-0ac0-48ea-a179-3e50d3e1729b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:59.490272 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23480248-0ac0-48ea-a179-3e50d3e1729b None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:59.491208 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23480248-0ac0-48ea-a179-3e50d3e1729b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:14:59.491747 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23480248-0ac0-48ea-a179-3e50d3e1729b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:59.493354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23480248-0ac0-48ea-a179-3e50d3e1729b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:59.503367 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 466/917] 10.222.0.47 () {56 vars in 1114 bytes} [Thu Jun 30 10:14:59 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:14:59.506449 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd22d9b-6e6b-4bc1-9363-d4ee09cdb467 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:59.597583 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:14:59.617266 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd22d9b-6e6b-4bc1-9363-d4ee09cdb467 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:59.638295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac7087-4e1b-4a53-85f0-2570eb1facc1 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:59.638587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac7087-4e1b-4a53-85f0-2570eb1facc1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:59.638853 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac7087-4e1b-4a53-85f0-2570eb1facc1 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:59.640517 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 467/918] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:14:59.646042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68877e-9e52-4a92-8a01-faf2e1acc804 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:59.646276 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68877e-9e52-4a92-8a01-faf2e1acc804 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:59.646546 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da68877e-9e52-4a92-8a01-faf2e1acc804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:59.657574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f2e2a-b251-418f-a799-ad35fb7834e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:59.658574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f2e2a-b251-418f-a799-ad35fb7834e2 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:59.659082 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f2e2a-b251-418f-a799-ad35fb7834e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:59.659410 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600f2e2a-b251-418f-a799-ad35fb7834e2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:59.660173 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f2e2a-b251-418f-a799-ad35fb7834e2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:59.661551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600f2e2a-b251-418f-a799-ad35fb7834e2 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:59.666385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-da68877e-9e52-4a92-8a01-faf2e1acc804 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:14:59.668321 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-da68877e-9e52-4a92-8a01-faf2e1acc804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:14:59.735289 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 452/919] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jun 30 10:14:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:14:59.756107 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 468/920] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:14:59 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 10:14:59.783379 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f2e2a-b251-418f-a799-ad35fb7834e2 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:59.908753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600f2e2a-b251-418f-a799-ad35fb7834e2 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:14:59.945595 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4625436d-11b0-4e2a-87fd-da603134650d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:14:59.946418 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625436d-11b0-4e2a-87fd-da603134650d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:14:59.946709 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625436d-11b0-4e2a-87fd-da603134650d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:14:59.946992 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4625436d-11b0-4e2a-87fd-da603134650d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:14:59.947696 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4625436d-11b0-4e2a-87fd-da603134650d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:14:59.949116 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4625436d-11b0-4e2a-87fd-da603134650d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:14:59.962091 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-4625436d-11b0-4e2a-87fd-da603134650d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 10:14:59.962091 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:59.962091 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 10:14:59.962091 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 10:14:59.962091 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 10:14:59.962091 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 10:14:59.962091 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 10:14:59.962091 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:14:59.962091 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 10:14:59.962091 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:59.962091 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:14:59.962091 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:59.962091 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:14:59.962091 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:14:59.962091 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:14:59.962091 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:14:59.962091 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:14:59.963117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:14:59.963117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:14:59.963117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:14:59.963117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:14:59.963117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:14:59.963117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:14:59.963117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 10:14:59.963117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 10:14:59.963117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 10:14:59.963117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 10:14:59.963117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:14:59.963117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:14:59.963117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:14:59.963117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:14:59.963117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:14:59.963117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:14:59.964002 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:14:59.964002 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:14:59.964002 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:14:59.964002 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:14:59.964002 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:14:59.964002 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:14:59.964002 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:14:59.964002 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:14:59.964002 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:14:59.964002 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:14:59.964002 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 10:14:59.964002 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:14:59.964002 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 10:14:59.964002 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:14:59.964002 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 453/921] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jun 30 10:14:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:14:59.972079 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68877e-9e52-4a92-8a01-faf2e1acc804 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:14:59.992987 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:00.085548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da68877e-9e52-4a92-8a01-faf2e1acc804 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:00.124197 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6316e61-63b6-4a29-945d-f2b2998fac65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:00.124871 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6316e61-63b6-4a29-945d-f2b2998fac65 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:00.125155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6316e61-63b6-4a29-945d-f2b2998fac65 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:00.126107 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6316e61-63b6-4a29-945d-f2b2998fac65 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:00.126913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6316e61-63b6-4a29-945d-f2b2998fac65 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:00.127225 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6316e61-63b6-4a29-945d-f2b2998fac65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:00.128635 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6316e61-63b6-4a29-945d-f2b2998fac65 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:00.133834 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 469/922] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jun 30 10:14:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:00.139548 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4625436d-11b0-4e2a-87fd-da603134650d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:00.151103 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:00.255194 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4625436d-11b0-4e2a-87fd-da603134650d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:00.291803 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad934e9-25b0-496b-8625-3ec3eed1ea63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:00.292852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad934e9-25b0-496b-8625-3ec3eed1ea63 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:00.293342 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad934e9-25b0-496b-8625-3ec3eed1ea63 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:00.293822 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad934e9-25b0-496b-8625-3ec3eed1ea63 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:00.294719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad934e9-25b0-496b-8625-3ec3eed1ea63 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:00.295228 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad934e9-25b0-496b-8625-3ec3eed1ea63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:00.296807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad934e9-25b0-496b-8625-3ec3eed1ea63 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:00.302649 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 454/923] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jun 30 10:15:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:00.308625 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6316e61-63b6-4a29-945d-f2b2998fac65 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:00.343440 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dadc4d-401d-4455-93e8-daed5a611663 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:00.343950 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dadc4d-401d-4455-93e8-daed5a611663 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:00.344987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dadc4d-401d-4455-93e8-daed5a611663 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:00.347033 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 455/924] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:15:00.352283 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ace12-9369-4837-a81d-34f1f815ca79 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:00.352793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ace12-9369-4837-a81d-34f1f815ca79 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:00.353261 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498ace12-9369-4837-a81d-34f1f815ca79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:00.374082 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-498ace12-9369-4837-a81d-34f1f815ca79 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:00.375923 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-498ace12-9369-4837-a81d-34f1f815ca79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:00.398614 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:00.421796 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6316e61-63b6-4a29-945d-f2b2998fac65 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:00.448919 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:00.455316 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 456/925] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 10:15:00.472651 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a0085-303b-4f90-8b74-7618e30e41e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:00.473660 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a0085-303b-4f90-8b74-7618e30e41e1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:00.474175 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a0085-303b-4f90-8b74-7618e30e41e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:00.474662 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a0085-303b-4f90-8b74-7618e30e41e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:00.475754 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a0085-303b-4f90-8b74-7618e30e41e1 None admin] RBAC: Adding query filter params (interface=public, service_id=ab06172975b149319f28f282f4c45cce, region_id=RegionOne) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:00.476306 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a0085-303b-4f90-8b74-7618e30e41e1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:00.477955 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a0085-303b-4f90-8b74-7618e30e41e1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:00.480482 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ace12-9369-4837-a81d-34f1f815ca79 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:00.488564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 470/926] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Jun 30 10:15:00 2022] GET /identity/v3/endpoints?service_id=ab06172975b149319f28f282f4c45cce&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:00.494002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a0085-303b-4f90-8b74-7618e30e41e1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:00.607358 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a0085-303b-4f90-8b74-7618e30e41e1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:00.608143 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498ace12-9369-4837-a81d-34f1f815ca79 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:00.642915 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52073f28-8829-4674-bbac-2fbdcbd4b9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:00.643629 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52073f28-8829-4674-bbac-2fbdcbd4b9b7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:00.643918 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52073f28-8829-4674-bbac-2fbdcbd4b9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:00.644199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52073f28-8829-4674-bbac-2fbdcbd4b9b7 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:00.644891 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52073f28-8829-4674-bbac-2fbdcbd4b9b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:00.646275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52073f28-8829-4674-bbac-2fbdcbd4b9b7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:00.648371 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c89432-e904-4ba8-baf7-f2e3c2f872d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:00.649432 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c89432-e904-4ba8-baf7-f2e3c2f872d0 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:00.649891 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c89432-e904-4ba8-baf7-f2e3c2f872d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:00.650407 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c89432-e904-4ba8-baf7-f2e3c2f872d0 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:00.651237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c89432-e904-4ba8-baf7-f2e3c2f872d0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:00.652967 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c89432-e904-4ba8-baf7-f2e3c2f872d0 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:00.653091 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 471/927] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jun 30 10:15:00 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:15:00.666674 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-d3c89432-e904-4ba8-baf7-f2e3c2f872d0 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 10:15:00.666674 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:00.666674 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 10:15:00.666674 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 10:15:00.666674 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 10:15:00.666674 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 10:15:00.666674 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 10:15:00.666674 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:15:00.666674 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 10:15:00.666674 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:00.666674 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:15:00.666674 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:00.666674 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:00.666674 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:00.666674 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:00.666674 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:00.668029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:00.668029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:00.668029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:00.668029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:00.668029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:00.668029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:00.668029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:00.668029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 10:15:00.668029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 10:15:00.668029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 10:15:00.668029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 10:15:00.668029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:00.668029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:00.668029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:00.668029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:00.668029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:00.668886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:00.668886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:00.668886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:00.668886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:00.668886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:00.668886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:00.668886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:00.668886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:00.668886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:00.668886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:00.668886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:00.668886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 10:15:00.668886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:15:00.668886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 30 10:15:00.668886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:00.672083 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 457/928] 10.222.0.47 () {56 vars in 1132 bytes} [Thu Jun 30 10:15:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:15:00.678910 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52073f28-8829-4674-bbac-2fbdcbd4b9b7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:00.693765 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:00.793990 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52073f28-8829-4674-bbac-2fbdcbd4b9b7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:00.832217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d4ce4-e30f-4f90-bc3e-d02f2e1be044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:00.833275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d4ce4-e30f-4f90-bc3e-d02f2e1be044 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:00.833772 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d4ce4-e30f-4f90-bc3e-d02f2e1be044 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:00.834248 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d4ce4-e30f-4f90-bc3e-d02f2e1be044 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:00.835156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d4ce4-e30f-4f90-bc3e-d02f2e1be044 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:00.835627 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d4ce4-e30f-4f90-bc3e-d02f2e1be044 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:00.837175 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d4ce4-e30f-4f90-bc3e-d02f2e1be044 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:00.842513 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 472/929] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jun 30 10:15:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:00.848858 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c89432-e904-4ba8-baf7-f2e3c2f872d0 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:00.917162 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:00.959229 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c89432-e904-4ba8-baf7-f2e3c2f872d0 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:00.995853 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714ecf5-d836-4e56-9dc5-4f4b94fa2b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:00.996526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714ecf5-d836-4e56-9dc5-4f4b94fa2b79 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:00.996813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714ecf5-d836-4e56-9dc5-4f4b94fa2b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:00.997089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714ecf5-d836-4e56-9dc5-4f4b94fa2b79 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:00.997799 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e714ecf5-d836-4e56-9dc5-4f4b94fa2b79 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:00.998161 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e714ecf5-d836-4e56-9dc5-4f4b94fa2b79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:00.999622 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e714ecf5-d836-4e56-9dc5-4f4b94fa2b79 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:01.005805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 458/930] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jun 30 10:15:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:01.011456 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d4ce4-e30f-4f90-bc3e-d02f2e1be044 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:01.090249 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:01.115723 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d4ce4-e30f-4f90-bc3e-d02f2e1be044 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:01.119562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f4a27-5841-48d1-a5f0-291c06427685 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:01.120096 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f4a27-5841-48d1-a5f0-291c06427685 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:01.120581 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f4a27-5841-48d1-a5f0-291c06427685 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:01.122405 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 459/931] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:15:01.129215 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ceeed9-3b4b-4394-8f77-971fa3d1b65c None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:01.129778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ceeed9-3b4b-4394-8f77-971fa3d1b65c None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:01.130453 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ceeed9-3b4b-4394-8f77-971fa3d1b65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:01.150431 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-b7ceeed9-3b4b-4394-8f77-971fa3d1b65c None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:01.151987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-b7ceeed9-3b4b-4394-8f77-971fa3d1b65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:01.154173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28c64c-bb6c-4f75-b4f8-12b9c9357bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:01.154850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28c64c-bb6c-4f75-b4f8-12b9c9357bc1 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:01.155130 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28c64c-bb6c-4f75-b4f8-12b9c9357bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:01.155431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b28c64c-bb6c-4f75-b4f8-12b9c9357bc1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:01.156651 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b28c64c-bb6c-4f75-b4f8-12b9c9357bc1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:01.158662 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b28c64c-bb6c-4f75-b4f8-12b9c9357bc1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:01.231413 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 460/932] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 10:15:01.250095 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ceeed9-3b4b-4394-8f77-971fa3d1b65c None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:01.311452 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 473/933] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jun 30 10:15:01 2022] POST /identity/v3/endpoints => generated 347 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:15:01.318841 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:01.373073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ceeed9-3b4b-4394-8f77-971fa3d1b65c None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:01.422580 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c9128-f770-449d-94ce-df359c340048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:01.423993 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c9128-f770-449d-94ce-df359c340048 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:01.424485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c9128-f770-449d-94ce-df359c340048 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:01.424960 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c9128-f770-449d-94ce-df359c340048 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:01.438460 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c9128-f770-449d-94ce-df359c340048 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:15:01.447547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-df1c9128-f770-449d-94ce-df359c340048 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 10:15:01.447547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:01.447547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:01.447547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:01.447547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:01.447547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:01.447547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:01.447547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:01.447547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:01.447547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:01.447547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:01.447547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:01.447547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:15:01.447547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:15:01.447547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:15:01.447547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:15:01.448544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:15:01.448544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:15:01.448544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:15:01.448544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:15:01.448544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:01.448544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:01.448544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:01.448544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:01.448544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:01.448544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:01.448544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:01.448544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:01.448544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:01.448544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:01.448544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:01.448544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:01.449384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:01.449384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:01.449384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:01.449384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:01.449384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:15:01.449384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:15:01.449384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:15:01.449384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:15:01.449384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:15:01.449384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:15:01.449384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 10:15:01.449384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:01.452390 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 461/934] 10.222.0.47 () {56 vars in 1120 bytes} [Thu Jun 30 10:15:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:15:01.459320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28c64c-bb6c-4f75-b4f8-12b9c9357bc1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:01.578387 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b28c64c-bb6c-4f75-b4f8-12b9c9357bc1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:01.608440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:01.615320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38836bd-b127-4ee0-9db1-3a355c288e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:01.615998 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38836bd-b127-4ee0-9db1-3a355c288e27 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:01.616280 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38836bd-b127-4ee0-9db1-3a355c288e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:01.616555 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38836bd-b127-4ee0-9db1-3a355c288e27 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:01.617299 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38836bd-b127-4ee0-9db1-3a355c288e27 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:01.617605 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38836bd-b127-4ee0-9db1-3a355c288e27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:01.619057 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38836bd-b127-4ee0-9db1-3a355c288e27 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:01.628754 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 474/935] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Jun 30 10:15:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:01.634351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c9128-f770-449d-94ce-df359c340048 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:01.744031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c9128-f770-449d-94ce-df359c340048 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:01.759342 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:01.781087 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cfb5d3-f3ad-4564-bdf7-21f423bb69c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:01.782751 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfb5d3-f3ad-4564-bdf7-21f423bb69c8 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:01.783201 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfb5d3-f3ad-4564-bdf7-21f423bb69c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:01.785628 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cfb5d3-f3ad-4564-bdf7-21f423bb69c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:01.795069 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cfb5d3-f3ad-4564-bdf7-21f423bb69c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:01.796788 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cfb5d3-f3ad-4564-bdf7-21f423bb69c8 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:01.814775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-d6cfb5d3-f3ad-4564-bdf7-21f423bb69c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:15:01.814775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:01.814775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:01.814775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:01.814775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:01.814775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:01.814775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:01.814775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:01.814775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:01.814775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:01.814775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:01.814775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:01.814775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:15:01.814775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:15:01.814775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:15:01.815838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:15:01.815838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:01.815838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:01.815838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:01.815838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:01.815838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:01.815838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:01.815838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:01.815838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:01.815838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:01.815838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:01.815838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:01.815838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:01.815838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:01.815838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:01.815838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:01.815838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:01.816717 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:15:01.816717 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:15:01.816717 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:15:01.816717 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:01.818763 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 462/936] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:15:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:15:01.824560 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38836bd-b127-4ee0-9db1-3a355c288e27 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:01.884615 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3559e-720c-4a0a-aae7-f09510b370ff None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:01.884906 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3559e-720c-4a0a-aae7-f09510b370ff None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:01.885188 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba3559e-720c-4a0a-aae7-f09510b370ff None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:01.887352 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 463/937] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:15:01.892328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8870f-e1de-42bc-957b-64a5d7a86b60 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:01.892621 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8870f-e1de-42bc-957b-64a5d7a86b60 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:01.892904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a8870f-e1de-42bc-957b-64a5d7a86b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:01.911224 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-a2a8870f-e1de-42bc-957b-64a5d7a86b60 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:01.912474 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-a2a8870f-e1de-42bc-957b-64a5d7a86b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:01.939428 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38836bd-b127-4ee0-9db1-3a355c288e27 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:01.978207 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2063a1-9afe-404e-b80a-23f991b7358b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:01.978881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2063a1-9afe-404e-b80a-23f991b7358b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:01.979170 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2063a1-9afe-404e-b80a-23f991b7358b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:01.979496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2063a1-9afe-404e-b80a-23f991b7358b None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:01.980232 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2063a1-9afe-404e-b80a-23f991b7358b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:01.980545 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2063a1-9afe-404e-b80a-23f991b7358b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:01.981929 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2063a1-9afe-404e-b80a-23f991b7358b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:01.988207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 464/938] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 10:15:02.004549 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 475/939] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:02.011359 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a8870f-e1de-42bc-957b-64a5d7a86b60 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:02.016769 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2063a1-9afe-404e-b80a-23f991b7358b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:02.081713 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:02.118678 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:02.126787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a8870f-e1de-42bc-957b-64a5d7a86b60 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:02.130122 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2063a1-9afe-404e-b80a-23f991b7358b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:02.165894 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744de32-49c3-415f-b850-e8ae8b2bf2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:02.166102 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbf257-4c05-450c-898c-7f3205ecc2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:02.166571 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbf257-4c05-450c-898c-7f3205ecc2d1 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:02.166863 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbf257-4c05-450c-898c-7f3205ecc2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:02.167140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbf257-4c05-450c-898c-7f3205ecc2d1 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:02.167891 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bbf257-4c05-450c-898c-7f3205ecc2d1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:02.168135 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744de32-49c3-415f-b850-e8ae8b2bf2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:02.168627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744de32-49c3-415f-b850-e8ae8b2bf2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:02.169122 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744de32-49c3-415f-b850-e8ae8b2bf2e5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:02.169335 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bbf257-4c05-450c-898c-7f3205ecc2d1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:02.170763 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: WARNING keystone.server.flask.common [None req-29bbf257-4c05-450c-898c-7f3205ecc2d1 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 30 10:15:02.178601 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3744de32-49c3-415f-b850-e8ae8b2bf2e5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:02.180729 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3744de32-49c3-415f-b850-e8ae8b2bf2e5 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:02.197578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-3744de32-49c3-415f-b850-e8ae8b2bf2e5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 10:15:02.197578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:02.197578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:02.197578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:02.197578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:02.197578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:02.197578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:02.197578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:02.197578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:02.197578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:02.197578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:02.197578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:02.197578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:15:02.197578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:15:02.197578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:15:02.197578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:15:02.198593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:02.198593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:02.198593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:15:02.198593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:15:02.198593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:15:02.198593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:15:02.198593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:02.198593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:02.198593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:02.198593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:02.198593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:02.198593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:02.198593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:02.198593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:02.198593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:02.198593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:02.198593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:02.198593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:02.199574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:02.199574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:02.199574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:15:02.199574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:15:02.199574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:15:02.199574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:15:02.199574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:15:02.199574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:15:02.199574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 10:15:02.199574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:02.203404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 465/940] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jun 30 10:15:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:15:02.205877 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744de32-49c3-415f-b850-e8ae8b2bf2e5 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:02.238522 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.sql.core [None req-29bbf257-4c05-450c-898c-7f3205ecc2d1 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 10:15:02.238522 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 30 10:15:02.238522 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [parameters: {'id': '8e1586a06cb144a38e0ba4e780520192', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 30 10:15:02.238522 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80363) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 10:15:02.262166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-29bbf257-4c05-450c-898c-7f3205ecc2d1 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8e1586a06cb144a38e0ba4e780520192.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8e1586a06cb144a38e0ba4e780520192. Jun 30 10:15:02.262166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:02.262166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 10:15:02.262166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 10:15:02.262166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 10:15:02.262166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 10:15:02.262166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 10:15:02.262166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 10:15:02.262166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 10:15:02.262166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application conn.query(q) Jun 30 10:15:02.262166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 10:15:02.262166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 10:15:02.262166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 10:15:02.262999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result.read() Jun 30 10:15:02.262999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 10:15:02.262999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 10:15:02.262999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 10:15:02.262999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 10:15:02.262999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 10:15:02.262999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 10:15:02.262999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 10:15:02.262999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 10:15:02.262999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 10:15:02.262999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:02.262999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 10:15:02.262999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:02.262999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:02.262999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 10:15:02.262999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 10:15:02.262999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 30 10:15:02.262999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 30 10:15:02.264004 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 10:15:02.264004 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application next(self.gen) Jun 30 10:15:02.264004 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 10:15:02.264004 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application yield resource Jun 30 10:15:02.264004 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 10:15:02.264004 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application next(self.gen) Jun 30 10:15:02.264004 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 10:15:02.264004 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 10:15:02.264004 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 10:15:02.264004 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 10:15:02.264004 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 10:15:02.264004 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise self.value Jun 30 10:15:02.264004 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 10:15:02.264004 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 10:15:02.264004 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 10:15:02.264004 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application session.commit() Jun 30 10:15:02.264004 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 10:15:02.264904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 10:15:02.264904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 10:15:02.264904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 10:15:02.264904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 10:15:02.264904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.session.flush() Jun 30 10:15:02.264904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 10:15:02.264904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 10:15:02.264904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 10:15:02.264904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 10:15:02.264904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 10:15:02.264904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application compat.raise_( Jun 30 10:15:02.264904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 10:15:02.264904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception Jun 30 10:15:02.264904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 10:15:02.264904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 10:15:02.264904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 10:15:02.264904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 10:15:02.265841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 10:15:02.265841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 10:15:02.265841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 10:15:02.265841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 10:15:02.265841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 30 10:15:02.265841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 30 10:15:02.265841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 10:15:02.265841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 10:15:02.265841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 10:15:02.265841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 10:15:02.265841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 10:15:02.265841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 10:15:02.265841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 10:15:02.265841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 10:15:02.265841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 10:15:02.267234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 10:15:02.267234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 10:15:02.267234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception Jun 30 10:15:02.267234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 10:15:02.267234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 10:15:02.267234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 10:15:02.267234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 10:15:02.267234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 10:15:02.267234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 10:15:02.267234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 10:15:02.267234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application conn.query(q) Jun 30 10:15:02.267234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 10:15:02.267234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 10:15:02.267234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 10:15:02.267234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result.read() Jun 30 10:15:02.267234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 10:15:02.267234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 10:15:02.268440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 10:15:02.268440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 10:15:02.268440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 10:15:02.268440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 10:15:02.268440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 10:15:02.268440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 10:15:02.268440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 30 10:15:02.268440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 30 10:15:02.268440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [parameters: {'id': '8e1586a06cb144a38e0ba4e780520192', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 30 10:15:02.268440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 10:15:02.268440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:02.268440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:15:02.268440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:02.268440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:02.268440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 30 10:15:02.271702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 30 10:15:02.271702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 10:15:02.271702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 10:15:02.271702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 30 10:15:02.271702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:02.271702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:15:02.271702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:02.271702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:02.271702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:02.271702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:02.271702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:02.271702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:02.271702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:02.271702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:02.271702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:02.271702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:02.271702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:02.272945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:02.272945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 30 10:15:02.272945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 30 10:15:02.272945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:02.272945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:02.272945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 30 10:15:02.272945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 30 10:15:02.272945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8e1586a06cb144a38e0ba4e780520192. Jun 30 10:15:02.272945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:02.272945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 476/941] 10.222.0.47 () {60 vars in 1140 bytes} [Thu Jun 30 10:15:02 2022] POST /identity/v3/projects => generated 262 bytes in 254 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:15:02.278160 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbf257-4c05-450c-898c-7f3205ecc2d1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:02.345287 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744de32-49c3-415f-b850-e8ae8b2bf2e5 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:02.383875 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e498d8ee-ef52-4ac1-9abd-ab255401c033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:02.385121 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498d8ee-ef52-4ac1-9abd-ab255401c033 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:02.385672 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498d8ee-ef52-4ac1-9abd-ab255401c033 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:02.386150 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498d8ee-ef52-4ac1-9abd-ab255401c033 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:02.387208 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e498d8ee-ef52-4ac1-9abd-ab255401c033 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:02.387890 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e498d8ee-ef52-4ac1-9abd-ab255401c033 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:02.389621 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e498d8ee-ef52-4ac1-9abd-ab255401c033 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:02.398103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbf257-4c05-450c-898c-7f3205ecc2d1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:02.407929 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:02.411940 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 466/942] 10.222.0.47 () {56 vars in 1149 bytes} [Thu Jun 30 10:15:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:02.418007 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e498d8ee-ef52-4ac1-9abd-ab255401c033 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:02.439290 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:02.445983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e07ee-5a9f-4df7-b78f-10f53d672235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:02.446734 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e07ee-5a9f-4df7-b78f-10f53d672235 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:02.447018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e07ee-5a9f-4df7-b78f-10f53d672235 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:02.447318 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e07ee-5a9f-4df7-b78f-10f53d672235 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:02.457234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e07ee-5a9f-4df7-b78f-10f53d672235 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:02.459640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e07ee-5a9f-4df7-b78f-10f53d672235 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:02.478626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-1a6e07ee-5a9f-4df7-b78f-10f53d672235 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:15:02.478626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:02.478626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:02.478626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:02.478626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:02.478626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:02.478626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:02.478626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:02.478626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:02.478626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:02.478626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:02.478626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:02.478626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:15:02.478626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:15:02.478626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:15:02.479649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:15:02.479649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:02.479649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:02.479649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:02.479649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:02.479649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:02.479649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:02.479649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:02.479649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:02.479649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:02.479649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:02.479649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:02.479649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:02.479649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:02.479649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:02.479649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:02.479649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:02.480535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:15:02.480535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:15:02.480535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:15:02.480535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:15:02.480535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:15:02.480535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:15:02.480535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:15:02.480535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:02.486168 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 477/943] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:15:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:15:02.487537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e07ee-5a9f-4df7-b78f-10f53d672235 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:02.561577 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e498d8ee-ef52-4ac1-9abd-ab255401c033 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:02.608027 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c92f4f-5635-42f5-82a8-f4acd5d4ff06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:02.609073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c92f4f-5635-42f5-82a8-f4acd5d4ff06 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:02.609503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c92f4f-5635-42f5-82a8-f4acd5d4ff06 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:02.610495 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c92f4f-5635-42f5-82a8-f4acd5d4ff06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:02.623206 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c92f4f-5635-42f5-82a8-f4acd5d4ff06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:02.624678 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c92f4f-5635-42f5-82a8-f4acd5d4ff06 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:02.635738 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e07ee-5a9f-4df7-b78f-10f53d672235 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:02.651125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-52c92f4f-5635-42f5-82a8-f4acd5d4ff06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:15:02.651125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:02.651125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:02.651125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:02.651125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:02.651125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:02.651125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:02.651125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:02.651125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:02.651125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:02.651125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:02.651125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:02.651125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:15:02.651125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:15:02.651125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:15:02.652104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:15:02.652104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:02.652104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:02.652104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:02.652104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:02.652104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:02.652104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:02.652104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:02.652104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:02.652104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:02.652104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:02.652104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:02.652104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:02.652104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:02.652104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:02.652104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:02.652104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:02.652978 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:15:02.652978 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:15:02.652978 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:15:02.652978 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:02.655852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 467/944] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:15:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:15:02.663287 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c92f4f-5635-42f5-82a8-f4acd5d4ff06 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:02.688717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5d74e-1b1a-499d-b5ab-35934a12c6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:02.689804 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5d74e-1b1a-499d-b5ab-35934a12c6bb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:02.690227 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5d74e-1b1a-499d-b5ab-35934a12c6bb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:02.690634 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5d74e-1b1a-499d-b5ab-35934a12c6bb None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:02.691451 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5d74e-1b1a-499d-b5ab-35934a12c6bb None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:02.692037 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5d74e-1b1a-499d-b5ab-35934a12c6bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:02.694189 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5d74e-1b1a-499d-b5ab-35934a12c6bb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:02.718706 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 478/945] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jun 30 10:15:02 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:02.743845 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:02.781174 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c92f4f-5635-42f5-82a8-f4acd5d4ff06 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:02.800187 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47df35-0056-486b-8bf2-37c6c6c8ffdb None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:02.800520 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47df35-0056-486b-8bf2-37c6c6c8ffdb None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:02.800754 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a47df35-0056-486b-8bf2-37c6c6c8ffdb None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:02.802155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 479/946] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:15:02.807899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557454f-4215-4ee9-935d-4a14b5dc24ae None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:02.808184 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557454f-4215-4ee9-935d-4a14b5dc24ae None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:02.808461 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9557454f-4215-4ee9-935d-4a14b5dc24ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:02.823412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7c064-271f-4d70-82cf-2065a98c4778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:02.824013 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7c064-271f-4d70-82cf-2065a98c4778 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:02.824300 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7c064-271f-4d70-82cf-2065a98c4778 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:02.824580 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a7c064-271f-4d70-82cf-2065a98c4778 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:02.825384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7c064-271f-4d70-82cf-2065a98c4778 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:02.825693 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7c064-271f-4d70-82cf-2065a98c4778 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:02.827139 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a7c064-271f-4d70-82cf-2065a98c4778 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:02.827247 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9557454f-4215-4ee9-935d-4a14b5dc24ae None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:02.828936 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9557454f-4215-4ee9-935d-4a14b5dc24ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:02.841003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 468/947] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:02.846952 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7c064-271f-4d70-82cf-2065a98c4778 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:02.864041 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:02.931653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 480/948] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:02 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 10:15:02.947803 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557454f-4215-4ee9-935d-4a14b5dc24ae None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:02.970206 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a7c064-271f-4d70-82cf-2065a98c4778 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:03.002868 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:03.017356 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8216e8-da53-4645-8906-7fa7ede2004a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:03.029830 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8216e8-da53-4645-8906-7fa7ede2004a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:03.029830 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8216e8-da53-4645-8906-7fa7ede2004a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:03.029830 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8216e8-da53-4645-8906-7fa7ede2004a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:03.029830 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8216e8-da53-4645-8906-7fa7ede2004a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:03.029830 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8216e8-da53-4645-8906-7fa7ede2004a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:03.042702 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-cc8216e8-da53-4645-8906-7fa7ede2004a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:15:03.042702 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:03.042702 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:03.042702 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:03.042702 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:03.042702 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:03.042702 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:03.042702 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:03.042702 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:03.042702 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:03.042702 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:03.042702 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:03.042702 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:15:03.042702 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:15:03.042702 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:15:03.043916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:15:03.043916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:03.043916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:03.043916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:03.043916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:03.043916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:03.043916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:03.043916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:03.043916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:03.043916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:03.043916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:03.043916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:03.043916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:03.043916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:03.043916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:03.043916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:03.043916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:03.044860 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:15:03.044860 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:15:03.044860 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:15:03.044860 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:15:03.044860 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:15:03.044860 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:15:03.044860 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:15:03.044860 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:03.044860 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 469/949] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:15:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:15:03.051039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8216e8-da53-4645-8906-7fa7ede2004a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:03.067889 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9557454f-4215-4ee9-935d-4a14b5dc24ae None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:03.104305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398c669-b2e7-4882-8258-804ec13c3b20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:03.105251 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398c669-b2e7-4882-8258-804ec13c3b20 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:03.105597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398c669-b2e7-4882-8258-804ec13c3b20 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:03.105939 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8398c669-b2e7-4882-8258-804ec13c3b20 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:03.106769 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8398c669-b2e7-4882-8258-804ec13c3b20 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:03.108804 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8398c669-b2e7-4882-8258-804ec13c3b20 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:03.122551 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-8398c669-b2e7-4882-8258-804ec13c3b20 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 10:15:03.122551 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:03.122551 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 10:15:03.122551 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 10:15:03.122551 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 10:15:03.122551 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 10:15:03.122551 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 10:15:03.122551 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:15:03.122551 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 10:15:03.122551 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:03.122551 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:15:03.122551 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:03.122551 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:03.122551 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:03.122551 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:03.122551 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:03.122551 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:03.123820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:03.123820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:03.123820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:03.123820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:03.123820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:03.123820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:03.123820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 10:15:03.123820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 10:15:03.123820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 10:15:03.123820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 10:15:03.123820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:03.123820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:03.123820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:03.123820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:03.123820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:03.123820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:03.124896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:03.124896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:03.124896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:03.124896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:03.124896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:03.124896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:03.124896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:03.124896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:03.124896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:03.124896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:03.124896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 10:15:03.124896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:15:03.124896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 10:15:03.124896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:03.124896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 481/950] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jun 30 10:15:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:15:03.131368 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398c669-b2e7-4882-8258-804ec13c3b20 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:03.158396 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:03.184735 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8216e8-da53-4645-8906-7fa7ede2004a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:03.228269 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155c2781-24a9-4128-9303-8ccb701fc33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:03.228973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c2781-24a9-4128-9303-8ccb701fc33e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:03.229425 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c2781-24a9-4128-9303-8ccb701fc33e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:03.229711 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155c2781-24a9-4128-9303-8ccb701fc33e None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:03.230899 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155c2781-24a9-4128-9303-8ccb701fc33e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:03.231498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155c2781-24a9-4128-9303-8ccb701fc33e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:03.233162 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155c2781-24a9-4128-9303-8ccb701fc33e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:03.248813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 470/951] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:15:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:03.249296 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8398c669-b2e7-4882-8258-804ec13c3b20 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:03.256790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155c2781-24a9-4128-9303-8ccb701fc33e None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:03.291583 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6fb5d-b999-4fee-a3a3-ecad01f68a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:03.292282 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6fb5d-b999-4fee-a3a3-ecad01f68a7a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:03.292569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6fb5d-b999-4fee-a3a3-ecad01f68a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:03.292850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6fb5d-b999-4fee-a3a3-ecad01f68a7a None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:03.293565 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6fb5d-b999-4fee-a3a3-ecad01f68a7a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:03.293952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6fb5d-b999-4fee-a3a3-ecad01f68a7a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:03.295251 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6fb5d-b999-4fee-a3a3-ecad01f68a7a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:03.305717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 482/952] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jun 30 10:15:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:03.307015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6fb5d-b999-4fee-a3a3-ecad01f68a7a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:03.317553 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:03.378732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155c2781-24a9-4128-9303-8ccb701fc33e None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:03.400300 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:03.415446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6fb5d-b999-4fee-a3a3-ecad01f68a7a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:03.418215 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad8c12e-4b82-4f8d-9856-f717ec563442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:03.419471 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8c12e-4b82-4f8d-9856-f717ec563442 None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:03.419959 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8c12e-4b82-4f8d-9856-f717ec563442 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:03.420441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad8c12e-4b82-4f8d-9856-f717ec563442 None admin] PATH_INFO: `/v3/projects/50902e0539434c82b505dc5009aeaf85/users/66ad8defb9b6453c96f1a8a7343695ba/roles/0b2dccc298c3428f91cea20447956c6a` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:03.423769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad8c12e-4b82-4f8d-9856-f717ec563442 None admin] RBAC: Authorizing `identity:create_grant(project_id=50902e0539434c82b505dc5009aeaf85, user_id=66ad8defb9b6453c96f1a8a7343695ba, role_id=0b2dccc298c3428f91cea20447956c6a)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:03.425536 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad8c12e-4b82-4f8d-9856-f717ec563442 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:03.454791 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d993f5-373a-4218-be44-d1de8a185aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:03.455606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d993f5-373a-4218-be44-d1de8a185aae None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:03.455895 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d993f5-373a-4218-be44-d1de8a185aae None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:03.456174 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d993f5-373a-4218-be44-d1de8a185aae None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:03.456891 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d993f5-373a-4218-be44-d1de8a185aae None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:03.457201 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d993f5-373a-4218-be44-d1de8a185aae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:03.458571 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d993f5-373a-4218-be44-d1de8a185aae None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:03.466031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 483/953] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jun 30 10:15:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:03.469418 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d993f5-373a-4218-be44-d1de8a185aae None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:03.516838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 471/954] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jun 30 10:15:03 2022] PUT /identity/v3/projects/50902e0539434c82b505dc5009aeaf85/users/66ad8defb9b6453c96f1a8a7343695ba/roles/0b2dccc298c3428f91cea20447956c6a => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:15:03.539676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a438a-dc5a-4026-8284-7fa1a0de944e None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:03.539988 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a438a-dc5a-4026-8284-7fa1a0de944e None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:03.540245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a438a-dc5a-4026-8284-7fa1a0de944e None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:03.541748 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 472/955] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:15:03.549137 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e8a88-8108-4227-9289-19ea1f02356c None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:03.549789 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e8a88-8108-4227-9289-19ea1f02356c None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:03.550262 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3e8a88-8108-4227-9289-19ea1f02356c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:03.572406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-2f3e8a88-8108-4227-9289-19ea1f02356c None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:03.573669 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-2f3e8a88-8108-4227-9289-19ea1f02356c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:03.604297 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d993f5-373a-4218-be44-d1de8a185aae None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:03.644402 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17381cba-bb44-40c6-8581-52b54938d037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:03.645440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17381cba-bb44-40c6-8581-52b54938d037 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:03.645959 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17381cba-bb44-40c6-8581-52b54938d037 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:03.646449 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17381cba-bb44-40c6-8581-52b54938d037 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:03.647690 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17381cba-bb44-40c6-8581-52b54938d037 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:03.649315 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17381cba-bb44-40c6-8581-52b54938d037 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:03.677489 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 473/956] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 10:15:03.700647 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e8a88-8108-4227-9289-19ea1f02356c None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:03.735754 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 484/957] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jun 30 10:15:03 2022] POST /identity/v3/endpoints => generated 349 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:15:03.740516 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:03.816639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3e8a88-8108-4227-9289-19ea1f02356c None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:03.866856 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad46adf-3f02-4f0e-ba35-f122f1b840aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:03.867697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad46adf-3f02-4f0e-ba35-f122f1b840aa None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:03.867980 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad46adf-3f02-4f0e-ba35-f122f1b840aa None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:03.868251 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad46adf-3f02-4f0e-ba35-f122f1b840aa None admin] PATH_INFO: `/v3/services/compute` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:03.868913 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad46adf-3f02-4f0e-ba35-f122f1b840aa None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:03.870323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad46adf-3f02-4f0e-ba35-f122f1b840aa None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:03.884554 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-cad46adf-3f02-4f0e-ba35-f122f1b840aa None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 10:15:03.884554 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:03.884554 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 10:15:03.884554 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 10:15:03.884554 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 10:15:03.884554 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 10:15:03.884554 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 10:15:03.884554 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:15:03.884554 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 10:15:03.884554 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:03.884554 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:15:03.884554 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:03.884554 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:03.884554 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:03.884554 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:03.884554 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:03.884554 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:03.885764 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:03.885764 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:03.885764 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:03.885764 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:03.885764 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:03.885764 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:03.885764 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 10:15:03.885764 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 10:15:03.885764 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 10:15:03.885764 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 10:15:03.885764 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:03.885764 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:03.885764 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:03.885764 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:03.885764 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:03.885764 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:03.886645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:03.886645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:03.886645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:03.886645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:03.886645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:03.886645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:03.886645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:03.886645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:03.886645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:03.886645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:03.886645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 10:15:03.886645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:15:03.886645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 10:15:03.886645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:03.886645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 474/958] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:15:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:15:03.892628 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17381cba-bb44-40c6-8581-52b54938d037 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:03.904930 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:04.009326 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17381cba-bb44-40c6-8581-52b54938d037 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:04.044849 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84304dfa-57a9-4a5d-a969-12c2c1a4558b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:04.045604 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84304dfa-57a9-4a5d-a969-12c2c1a4558b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:04.045891 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84304dfa-57a9-4a5d-a969-12c2c1a4558b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:04.046222 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84304dfa-57a9-4a5d-a969-12c2c1a4558b None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:04.046963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84304dfa-57a9-4a5d-a969-12c2c1a4558b None admin] RBAC: Adding query filter params (name=compute) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:04.048618 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84304dfa-57a9-4a5d-a969-12c2c1a4558b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:04.050100 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84304dfa-57a9-4a5d-a969-12c2c1a4558b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:04.059376 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 485/959] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jun 30 10:15:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:04.061879 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad46adf-3f02-4f0e-ba35-f122f1b840aa None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:04.167404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:04.173768 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad46adf-3f02-4f0e-ba35-f122f1b840aa None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:04.212607 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c21e6-be14-4163-be85-a87ec5ea9eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:04.213275 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c21e6-be14-4163-be85-a87ec5ea9eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:04.213556 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c21e6-be14-4163-be85-a87ec5ea9eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:04.213830 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c21e6-be14-4163-be85-a87ec5ea9eb6 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:04.214570 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c21e6-be14-4163-be85-a87ec5ea9eb6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:04.214876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c21e6-be14-4163-be85-a87ec5ea9eb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:04.216261 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c21e6-be14-4163-be85-a87ec5ea9eb6 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:04.222597 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 475/960] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jun 30 10:15:04 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:05.010018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818b9a9-9ce8-485e-b3c4-b073470efbe7 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:05.010569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818b9a9-9ce8-485e-b3c4-b073470efbe7 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:05.010569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818b9a9-9ce8-485e-b3c4-b073470efbe7 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:05.012268 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 486/961] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:15:05.017533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d13d0-03da-4f7a-bf2d-4336b658eec4 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:05.017833 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d13d0-03da-4f7a-bf2d-4336b658eec4 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:05.018117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d13d0-03da-4f7a-bf2d-4336b658eec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:05.044344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-a15d13d0-03da-4f7a-bf2d-4336b658eec4 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:05.045572 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-a15d13d0-03da-4f7a-bf2d-4336b658eec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:05.145106 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 476/962] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 10:15:05.176322 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818b9a9-9ce8-485e-b3c4-b073470efbe7 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:05.260933 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:05.308318 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818b9a9-9ce8-485e-b3c4-b073470efbe7 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:05.357583 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46e2a9-bb7a-4728-a301-53935aa72aa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:05.358266 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46e2a9-bb7a-4728-a301-53935aa72aa9 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:05.358556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46e2a9-bb7a-4728-a301-53935aa72aa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:05.358837 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee46e2a9-bb7a-4728-a301-53935aa72aa9 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:05.359648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46e2a9-bb7a-4728-a301-53935aa72aa9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:05.361183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee46e2a9-bb7a-4728-a301-53935aa72aa9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:05.405764 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.sql.core [None req-ee46e2a9-bb7a-4728-a301-53935aa72aa9 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 10:15:05.405764 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 30 10:15:05.405764 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [parameters: {'user_id': 'fe4fdebef4164c97bc2315ca151ab4b9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 30 10:15:05.405764 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80363) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 30 10:15:05.413715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-ee46e2a9-bb7a-4728-a301-53935aa72aa9 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 30 10:15:05.413715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:05.413715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 10:15:05.413715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 10:15:05.413715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 10:15:05.413715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 10:15:05.413715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 10:15:05.413715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 10:15:05.413715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 10:15:05.413715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application conn.query(q) Jun 30 10:15:05.413715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 10:15:05.413715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 10:15:05.413715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 10:15:05.413715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result.read() Jun 30 10:15:05.413715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 10:15:05.414594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 10:15:05.414594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 10:15:05.414594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 10:15:05.414594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 10:15:05.414594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 10:15:05.414594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 10:15:05.414594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 10:15:05.414594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 10:15:05.414594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:05.414594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 30 10:15:05.414594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:05.414594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:05.414594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 30 10:15:05.414594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 30 10:15:05.414594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 30 10:15:05.414594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 30 10:15:05.414594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 10:15:05.414594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application next(self.gen) Jun 30 10:15:05.415761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 30 10:15:05.415761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application yield resource Jun 30 10:15:05.415761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 30 10:15:05.415761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application next(self.gen) Jun 30 10:15:05.415761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 30 10:15:05.415761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.session.rollback() Jun 30 10:15:05.415761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 30 10:15:05.415761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.force_reraise() Jun 30 10:15:05.415761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 30 10:15:05.415761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise self.value Jun 30 10:15:05.415761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 30 10:15:05.415761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 30 10:15:05.415761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 30 10:15:05.415761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application session.commit() Jun 30 10:15:05.415761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 30 10:15:05.415761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 30 10:15:05.415761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 30 10:15:05.416961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._prepare_impl() Jun 30 10:15:05.416961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 30 10:15:05.416961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.session.flush() Jun 30 10:15:05.416961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 30 10:15:05.416961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._flush(objects) Jun 30 10:15:05.416961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 30 10:15:05.416961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 30 10:15:05.416961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 30 10:15:05.416961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application compat.raise_( Jun 30 10:15:05.416961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 10:15:05.416961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception Jun 30 10:15:05.416961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 30 10:15:05.416961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application flush_context.execute() Jun 30 10:15:05.416961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 30 10:15:05.416961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rec.execute(self) Jun 30 10:15:05.416961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 30 10:15:05.416961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 30 10:15:05.417922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 30 10:15:05.417922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 30 10:15:05.417922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 30 10:15:05.417922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 30 10:15:05.417922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 30 10:15:05.417922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 30 10:15:05.417922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jun 30 10:15:05.417922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 30 10:15:05.417922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 30 10:15:05.417922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 30 10:15:05.417922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 30 10:15:05.417922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 30 10:15:05.417922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 30 10:15:05.417922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 30 10:15:05.417922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 30 10:15:05.417922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception Jun 30 10:15:05.418779 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 30 10:15:05.418779 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 30 10:15:05.418779 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 30 10:15:05.418779 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 30 10:15:05.418779 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 30 10:15:05.418779 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result = self._query(query) Jun 30 10:15:05.418779 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 30 10:15:05.418779 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application conn.query(q) Jun 30 10:15:05.418779 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 30 10:15:05.418779 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 30 10:15:05.418779 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 30 10:15:05.418779 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application result.read() Jun 30 10:15:05.418779 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 30 10:15:05.418779 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 30 10:15:05.418779 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 30 10:15:05.418779 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 30 10:15:05.419665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 30 10:15:05.419665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 30 10:15:05.419665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 30 10:15:05.419665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 30 10:15:05.419665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 30 10:15:05.419665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 30 10:15:05.419665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [parameters: {'user_id': 'fe4fdebef4164c97bc2315ca151ab4b9', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 30 10:15:05.419665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 30 10:15:05.419665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:05.419665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:15:05.419665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:05.419665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:05.419665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:05.419665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:05.419665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:05.420732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:05.420732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:05.420732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:05.420732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:05.420732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:05.420732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:05.420732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:05.420732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 30 10:15:05.420732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 30 10:15:05.420732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:05.420732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:05.420732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:15:05.420732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:15:05.420732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:15:05.420732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:15:05.420732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 30 10:15:05.420732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 30 10:15:05.422310 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 30 10:15:05.422310 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 30 10:15:05.422310 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 30 10:15:05.422310 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 30 10:15:05.422310 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 30 10:15:05.422310 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:05.422310 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 487/963] 10.222.0.47 () {60 vars in 1131 bytes} [Thu Jun 30 10:15:05 2022] POST /identity/v3/users => generated 160 bytes in 239 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:15:05.424394 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d13d0-03da-4f7a-bf2d-4336b658eec4 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:05.542394 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:05.552746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d13d0-03da-4f7a-bf2d-4336b658eec4 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:05.589325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73aea482-e4f6-4a64-89ce-c6ed4cd0dbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:05.590470 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aea482-e4f6-4a64-89ce-c6ed4cd0dbc0 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:05.591112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aea482-e4f6-4a64-89ce-c6ed4cd0dbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:05.591600 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73aea482-e4f6-4a64-89ce-c6ed4cd0dbc0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:05.599744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aea482-e4f6-4a64-89ce-c6ed4cd0dbc0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:05.601400 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73aea482-e4f6-4a64-89ce-c6ed4cd0dbc0 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:05.616386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-73aea482-e4f6-4a64-89ce-c6ed4cd0dbc0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 10:15:05.616386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:05.616386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:05.616386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:05.616386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:05.616386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:05.616386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:05.616386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:05.616386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:05.616386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:05.616386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:05.616386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:05.616386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:15:05.616386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:15:05.616386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:15:05.616386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:15:05.617344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:05.617344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:05.617344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:15:05.617344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:15:05.617344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:15:05.617344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:15:05.617344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:05.617344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:05.617344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:05.617344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:05.617344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:05.617344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:05.617344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:05.617344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:05.617344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:05.617344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:05.617344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:05.617344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:05.618376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:05.618376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:05.618376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:15:05.618376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:15:05.618376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:15:05.618376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:15:05.618376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:15:05.618376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:15:05.618376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 30 10:15:05.618376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:05.620801 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 477/964] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:15:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:15:05.631292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46e2a9-bb7a-4728-a301-53935aa72aa9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:05.714798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b13629-ddb6-437e-a717-e5d2ece42d48 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:05.715169 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b13629-ddb6-437e-a717-e5d2ece42d48 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:05.715394 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b13629-ddb6-437e-a717-e5d2ece42d48 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:05.717110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 478/965] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:15:05.723828 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d7ea8-4c03-4d03-8bc3-0108384c6c36 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:05.723976 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d7ea8-4c03-4d03-8bc3-0108384c6c36 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:05.724188 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741d7ea8-4c03-4d03-8bc3-0108384c6c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:05.741480 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-741d7ea8-4c03-4d03-8bc3-0108384c6c36 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:05.742454 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee46e2a9-bb7a-4728-a301-53935aa72aa9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:05.742656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-741d7ea8-4c03-4d03-8bc3-0108384c6c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:05.780954 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:05.787963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a297ba-7e7f-4126-8dc3-7c7edb0d9b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:05.789236 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a297ba-7e7f-4126-8dc3-7c7edb0d9b5f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:05.789977 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a297ba-7e7f-4126-8dc3-7c7edb0d9b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:05.790482 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a297ba-7e7f-4126-8dc3-7c7edb0d9b5f None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:05.791702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a297ba-7e7f-4126-8dc3-7c7edb0d9b5f None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:05.792642 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a297ba-7e7f-4126-8dc3-7c7edb0d9b5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:05.794977 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a297ba-7e7f-4126-8dc3-7c7edb0d9b5f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:05.823399 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 479/966] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 10:15:05.839239 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d7ea8-4c03-4d03-8bc3-0108384c6c36 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:05.840246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 488/967] 10.222.0.47 () {56 vars in 1101 bytes} [Thu Jun 30 10:15:05 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:05.955685 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:05.958570 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741d7ea8-4c03-4d03-8bc3-0108384c6c36 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:05.997389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26baf19-72b9-4edc-bf10-b9c7ecfdef8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:05.998209 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26baf19-72b9-4edc-bf10-b9c7ecfdef8c None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:05.998492 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26baf19-72b9-4edc-bf10-b9c7ecfdef8c None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:05.998767 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26baf19-72b9-4edc-bf10-b9c7ecfdef8c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:06.009773 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26baf19-72b9-4edc-bf10-b9c7ecfdef8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:15:06.018536 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-f26baf19-72b9-4edc-bf10-b9c7ecfdef8c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 10:15:06.018536 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:06.018536 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:06.018536 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:06.018536 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:06.018536 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:06.018536 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:06.018536 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:06.018536 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:06.018536 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:06.018536 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:06.018536 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:06.018536 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:15:06.018536 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:15:06.018536 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:15:06.018536 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:15:06.019533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:15:06.019533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:15:06.019533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:15:06.019533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:15:06.019533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:06.019533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:06.019533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:06.019533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:06.019533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:06.019533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:06.019533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:06.019533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:06.019533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:06.019533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:06.019533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:06.019533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:06.020389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:06.020389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:06.020389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:06.020389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:06.020389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:15:06.020389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:15:06.020389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:15:06.020389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:15:06.020389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:15:06.020389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:15:06.020389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 10:15:06.020389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:06.025008 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 480/968] 10.222.0.47 () {56 vars in 1120 bytes} [Thu Jun 30 10:15:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:15:06.026385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a297ba-7e7f-4126-8dc3-7c7edb0d9b5f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:06.114767 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e7d36-0e27-45ac-ab67-81dcf5d6d7c8 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:06.115009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e7d36-0e27-45ac-ab67-81dcf5d6d7c8 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:06.115304 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e7d36-0e27-45ac-ab67-81dcf5d6d7c8 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:06.120370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 481/969] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:15:06.121915 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de85933-1d4a-4a82-98fe-7e948b118bf8 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:06.122204 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de85933-1d4a-4a82-98fe-7e948b118bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:06.122483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de85933-1d4a-4a82-98fe-7e948b118bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:06.138913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a297ba-7e7f-4126-8dc3-7c7edb0d9b5f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:06.147507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-7de85933-1d4a-4a82-98fe-7e948b118bf8 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:06.148789 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-7de85933-1d4a-4a82-98fe-7e948b118bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:06.175419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f6773-0dd6-4d12-92a3-e71faae83c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:06.176138 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f6773-0dd6-4d12-92a3-e71faae83c66 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:06.176495 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f6773-0dd6-4d12-92a3-e71faae83c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:06.176778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4f6773-0dd6-4d12-92a3-e71faae83c66 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:06.177532 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4f6773-0dd6-4d12-92a3-e71faae83c66 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:06.177836 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4f6773-0dd6-4d12-92a3-e71faae83c66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:06.179259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4f6773-0dd6-4d12-92a3-e71faae83c66 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:06.188360 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 489/970] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Jun 30 10:15:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:06.194067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f6773-0dd6-4d12-92a3-e71faae83c66 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:06.223315 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 482/971] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 10:15:06.245068 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de85933-1d4a-4a82-98fe-7e948b118bf8 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:06.253534 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:06.305406 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4f6773-0dd6-4d12-92a3-e71faae83c66 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:06.321275 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:06.342780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001346b7-e6d9-4ce2-8908-8fa99f7dcf75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:06.343613 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001346b7-e6d9-4ce2-8908-8fa99f7dcf75 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:06.343995 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001346b7-e6d9-4ce2-8908-8fa99f7dcf75 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:06.344298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001346b7-e6d9-4ce2-8908-8fa99f7dcf75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:06.350367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de85933-1d4a-4a82-98fe-7e948b118bf8 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:06.355925 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001346b7-e6d9-4ce2-8908-8fa99f7dcf75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:06.357383 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001346b7-e6d9-4ce2-8908-8fa99f7dcf75 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:06.375969 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-001346b7-e6d9-4ce2-8908-8fa99f7dcf75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:15:06.375969 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:06.375969 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:06.375969 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:06.375969 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:06.375969 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:06.375969 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:06.375969 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:06.375969 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:06.375969 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:06.375969 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:06.375969 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:06.375969 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:15:06.375969 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:15:06.375969 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:15:06.376872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:15:06.376872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:06.376872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:06.376872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:06.376872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:06.376872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:06.376872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:06.376872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:06.376872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:06.376872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:06.376872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:06.376872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:06.376872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:06.376872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:06.376872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:06.376872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:06.376872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:06.377928 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:15:06.377928 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:15:06.377928 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:15:06.377928 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:06.377928 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 490/972] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:15:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:15:06.383390 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001346b7-e6d9-4ce2-8908-8fa99f7dcf75 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:06.395464 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324848c-0972-40c2-a49b-24de3770e460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:06.396623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324848c-0972-40c2-a49b-24de3770e460 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:06.397206 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324848c-0972-40c2-a49b-24de3770e460 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:06.397608 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f324848c-0972-40c2-a49b-24de3770e460 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:06.398281 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f324848c-0972-40c2-a49b-24de3770e460 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:06.399671 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f324848c-0972-40c2-a49b-24de3770e460 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:06.412237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-f324848c-0972-40c2-a49b-24de3770e460 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 10:15:06.412237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:06.412237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 10:15:06.412237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 10:15:06.412237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 10:15:06.412237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 10:15:06.412237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 10:15:06.412237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:15:06.412237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 10:15:06.412237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:06.412237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:15:06.412237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:06.412237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:06.412237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:06.412237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:06.412237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:06.412237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:06.413277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:06.413277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:06.413277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:06.413277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:06.413277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:06.413277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:06.413277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 10:15:06.413277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 10:15:06.413277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 10:15:06.413277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 10:15:06.413277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:06.413277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:06.413277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:06.413277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:06.413277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:06.413277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:06.414223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:06.414223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:06.414223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:06.414223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:06.414223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:06.414223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:06.414223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:06.414223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:06.414223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:06.414223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:06.414223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 10:15:06.414223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:15:06.414223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 10:15:06.414223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:06.414223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 483/973] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jun 30 10:15:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:15:06.419574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324848c-0972-40c2-a49b-24de3770e460 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:06.492232 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001346b7-e6d9-4ce2-8908-8fa99f7dcf75 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:06.494481 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:06.525581 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f324848c-0972-40c2-a49b-24de3770e460 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:06.527511 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af04b741-c93a-45b2-8690-196e979908fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:06.528179 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04b741-c93a-45b2-8690-196e979908fc None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:06.528496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04b741-c93a-45b2-8690-196e979908fc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:06.528775 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af04b741-c93a-45b2-8690-196e979908fc None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:06.529502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af04b741-c93a-45b2-8690-196e979908fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:06.529810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af04b741-c93a-45b2-8690-196e979908fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:06.531191 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af04b741-c93a-45b2-8690-196e979908fc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:06.542540 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 491/974] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:15:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:06.543968 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769007d-7a8e-413f-828e-8ea0f21a0ec4 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:06.544260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769007d-7a8e-413f-828e-8ea0f21a0ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:06.544540 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1769007d-7a8e-413f-828e-8ea0f21a0ec4 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:06.545770 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 492/975] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:15:06.548977 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769007d-7a8e-413f-828e-8ea0f21a0ec4 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:06.560131 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23fde76-6f9c-41f8-a7b2-a607ba7eab72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:06.560837 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fde76-6f9c-41f8-a7b2-a607ba7eab72 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:06.561094 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fde76-6f9c-41f8-a7b2-a607ba7eab72 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:06.561374 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23fde76-6f9c-41f8-a7b2-a607ba7eab72 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:06.562080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23fde76-6f9c-41f8-a7b2-a607ba7eab72 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:06.562392 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23fde76-6f9c-41f8-a7b2-a607ba7eab72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:06.564556 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23fde76-6f9c-41f8-a7b2-a607ba7eab72 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:06.565615 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:06.569748 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 484/976] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jun 30 10:15:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:06.570933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7af68-e397-4d45-ae49-281e89aaae38 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:06.571225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7af68-e397-4d45-ae49-281e89aaae38 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:06.571534 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e7af68-e397-4d45-ae49-281e89aaae38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:06.587409 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-d6e7af68-e397-4d45-ae49-281e89aaae38 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:06.588613 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d6e7af68-e397-4d45-ae49-281e89aaae38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:06.654604 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1769007d-7a8e-413f-828e-8ea0f21a0ec4 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:06.655524 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 485/977] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:15:06.656846 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7af68-e397-4d45-ae49-281e89aaae38 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:06.700971 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d3934-dd70-4ae3-9587-8af4b58eb4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:06.701781 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d3934-dd70-4ae3-9587-8af4b58eb4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:06.702067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d3934-dd70-4ae3-9587-8af4b58eb4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:06.702336 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d3934-dd70-4ae3-9587-8af4b58eb4f0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:06.709675 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d3934-dd70-4ae3-9587-8af4b58eb4f0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:06.711120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d3934-dd70-4ae3-9587-8af4b58eb4f0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:06.726182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-a66d3934-dd70-4ae3-9587-8af4b58eb4f0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 10:15:06.726182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:06.726182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:06.726182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:06.726182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:06.726182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:06.726182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:06.726182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:06.726182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:06.726182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:06.726182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:06.726182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:06.726182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:15:06.726182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:15:06.726182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:15:06.726182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:15:06.727154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:06.727154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:06.727154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:15:06.727154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:15:06.727154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:15:06.727154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:15:06.727154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:06.727154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:06.727154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:06.727154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:06.727154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:06.727154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:06.727154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:06.727154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:06.727154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:06.727154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:06.727154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:06.727154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:06.728140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:06.728140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:06.728140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:15:06.728140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:15:06.728140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:15:06.728140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:15:06.728140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:15:06.728140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:15:06.728140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 30 10:15:06.728140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:06.728140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 493/978] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jun 30 10:15:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:15:06.728140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d3934-dd70-4ae3-9587-8af4b58eb4f0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:06.755877 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e7af68-e397-4d45-ae49-281e89aaae38 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:06.757257 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:06.783254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:06.791599 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174a272-8d19-48ee-bd36-4d28cdc1067f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:06.792265 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174a272-8d19-48ee-bd36-4d28cdc1067f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:06.792550 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174a272-8d19-48ee-bd36-4d28cdc1067f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:06.792828 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a174a272-8d19-48ee-bd36-4d28cdc1067f None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:06.793532 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a174a272-8d19-48ee-bd36-4d28cdc1067f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:06.793873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a174a272-8d19-48ee-bd36-4d28cdc1067f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:06.795323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a174a272-8d19-48ee-bd36-4d28cdc1067f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:06.801073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 486/979] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jun 30 10:15:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:06.801762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174a272-8d19-48ee-bd36-4d28cdc1067f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:06.837899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d3934-dd70-4ae3-9587-8af4b58eb4f0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:06.872685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38576c3-63f4-4fac-b75c-2c1f6a2ff65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:06.873342 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38576c3-63f4-4fac-b75c-2c1f6a2ff65b None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:06.873625 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38576c3-63f4-4fac-b75c-2c1f6a2ff65b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:06.873902 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38576c3-63f4-4fac-b75c-2c1f6a2ff65b None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:06.874620 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38576c3-63f4-4fac-b75c-2c1f6a2ff65b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:06.876381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38576c3-63f4-4fac-b75c-2c1f6a2ff65b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:06.898800 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a174a272-8d19-48ee-bd36-4d28cdc1067f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:06.932629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c087b2-74a5-42fb-9ce7-573921f6ddfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:06.933301 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c087b2-74a5-42fb-9ce7-573921f6ddfe None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:06.933588 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c087b2-74a5-42fb-9ce7-573921f6ddfe None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:06.933868 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c087b2-74a5-42fb-9ce7-573921f6ddfe None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:06.934705 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c087b2-74a5-42fb-9ce7-573921f6ddfe None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:06.935018 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c087b2-74a5-42fb-9ce7-573921f6ddfe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:06.936484 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c087b2-74a5-42fb-9ce7-573921f6ddfe None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:06.952936 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 487/980] 10.222.0.47 () {56 vars in 1149 bytes} [Thu Jun 30 10:15:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:06.954249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c087b2-74a5-42fb-9ce7-573921f6ddfe None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:06.983123 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 494/981] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jun 30 10:15:06 2022] POST /identity/v3/services => generated 242 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:15:06.983935 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38576c3-63f4-4fac-b75c-2c1f6a2ff65b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:07.004936 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:07.061218 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c087b2-74a5-42fb-9ce7-573921f6ddfe None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:07.093533 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:07.112005 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38576c3-63f4-4fac-b75c-2c1f6a2ff65b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:07.112241 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfaedaa-47b2-4134-af5b-842680fee008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:07.112836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfaedaa-47b2-4134-af5b-842680fee008 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:07.113125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfaedaa-47b2-4134-af5b-842680fee008 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:07.113403 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfaedaa-47b2-4134-af5b-842680fee008 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:07.114229 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfaedaa-47b2-4134-af5b-842680fee008 None admin] RBAC: Adding query filter params (interface=admin, service_id=ab06172975b149319f28f282f4c45cce, region_id=RegionOne) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:07.115296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfaedaa-47b2-4134-af5b-842680fee008 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:07.116004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfaedaa-47b2-4134-af5b-842680fee008 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:07.127609 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 488/982] 10.222.0.47 () {56 vars in 1249 bytes} [Thu Jun 30 10:15:06 2022] GET /identity/v3/endpoints?service_id=ab06172975b149319f28f282f4c45cce&interface=admin®ion_id=RegionOne => generated 193 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:07.128761 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfaedaa-47b2-4134-af5b-842680fee008 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:07.150441 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198badaf-ab59-4fa9-ad07-ae62753028bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:07.151092 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198badaf-ab59-4fa9-ad07-ae62753028bd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:07.151405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198badaf-ab59-4fa9-ad07-ae62753028bd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:07.151692 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198badaf-ab59-4fa9-ad07-ae62753028bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:07.228688 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198badaf-ab59-4fa9-ad07-ae62753028bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:07.230067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198badaf-ab59-4fa9-ad07-ae62753028bd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:07.243527 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfaedaa-47b2-4134-af5b-842680fee008 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:07.255509 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:07.280724 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce8480-d038-4715-972f-32f52a78daa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:07.281406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce8480-d038-4715-972f-32f52a78daa7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:07.281772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce8480-d038-4715-972f-32f52a78daa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:07.282102 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ce8480-d038-4715-972f-32f52a78daa7 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:07.282793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce8480-d038-4715-972f-32f52a78daa7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:07.284315 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ce8480-d038-4715-972f-32f52a78daa7 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:07.290589 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 489/983] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jun 30 10:15:07 2022] GET /identity/v3/services => generated 1727 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:15:07.324021 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:07.346306 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 495/984] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:15:06 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 10:15:07.348575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce8480-d038-4715-972f-32f52a78daa7 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:07.462630 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ce8480-d038-4715-972f-32f52a78daa7 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:07.497795 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad244c-ba17-483b-a789-6a1d46646f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:07.498832 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad244c-ba17-483b-a789-6a1d46646f5f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:07.499256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad244c-ba17-483b-a789-6a1d46646f5f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:07.499700 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cad244c-ba17-483b-a789-6a1d46646f5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:07.511180 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cad244c-ba17-483b-a789-6a1d46646f5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:07.512675 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cad244c-ba17-483b-a789-6a1d46646f5f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:07.531679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-7cad244c-ba17-483b-a789-6a1d46646f5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:15:07.531679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:07.531679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:07.531679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:07.531679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:07.531679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:07.531679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:07.531679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:07.531679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:07.531679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:07.531679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:07.531679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:07.531679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:15:07.531679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:15:07.531679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:15:07.532627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:15:07.532627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:07.532627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:07.532627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:07.532627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:07.532627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:07.532627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:07.532627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:07.532627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:07.532627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:07.532627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:07.532627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:07.532627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:07.532627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:07.532627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:07.532627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:07.532627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:07.533604 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:15:07.533604 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:15:07.533604 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:15:07.533604 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:07.533604 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 490/985] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:15:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:15:07.538971 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198badaf-ab59-4fa9-ad07-ae62753028bd None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:07.645448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198badaf-ab59-4fa9-ad07-ae62753028bd None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:07.679537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719fb5fb-45b3-4634-85bb-bd5bbce34246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:07.680207 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719fb5fb-45b3-4634-85bb-bd5bbce34246 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:07.680493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719fb5fb-45b3-4634-85bb-bd5bbce34246 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:07.680771 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719fb5fb-45b3-4634-85bb-bd5bbce34246 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:07.681496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719fb5fb-45b3-4634-85bb-bd5bbce34246 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:07.681807 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719fb5fb-45b3-4634-85bb-bd5bbce34246 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:07.683189 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719fb5fb-45b3-4634-85bb-bd5bbce34246 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:07.699476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 496/986] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:15:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:07.701108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad244c-ba17-483b-a789-6a1d46646f5f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:07.732849 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:07.808700 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cad244c-ba17-483b-a789-6a1d46646f5f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:07.842812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b1032-fe07-4c15-a73f-a1ac53dfb04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:07.843640 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b1032-fe07-4c15-a73f-a1ac53dfb04e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:07.844065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b1032-fe07-4c15-a73f-a1ac53dfb04e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:07.844350 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7b1032-fe07-4c15-a73f-a1ac53dfb04e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:07.851136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7b1032-fe07-4c15-a73f-a1ac53dfb04e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:07.852613 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7b1032-fe07-4c15-a73f-a1ac53dfb04e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:07.867256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-ee7b1032-fe07-4c15-a73f-a1ac53dfb04e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:15:07.867256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:07.867256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:07.867256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:07.867256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:07.867256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:07.867256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:07.867256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:07.867256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:07.867256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:07.867256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:07.867256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:07.867256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:15:07.867256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:15:07.867256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:15:07.868191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:15:07.868191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:07.868191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:07.868191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:07.868191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:07.868191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:07.868191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:07.868191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:07.868191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:07.868191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:07.868191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:07.868191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:07.868191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:07.868191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:07.868191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:07.868191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:07.868191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:07.869113 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:15:07.869113 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:15:07.869113 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:15:07.869113 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:15:07.869113 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:15:07.869113 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:15:07.869113 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:15:07.869113 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:07.869113 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 491/987] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:15:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:15:07.874320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719fb5fb-45b3-4634-85bb-bd5bbce34246 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:07.925160 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:07.981279 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719fb5fb-45b3-4634-85bb-bd5bbce34246 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:08.015585 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73214993-db25-4656-96c9-d1fad7e67308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:08.016255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73214993-db25-4656-96c9-d1fad7e67308 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:08.016633 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73214993-db25-4656-96c9-d1fad7e67308 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:08.016918 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73214993-db25-4656-96c9-d1fad7e67308 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:08.017712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73214993-db25-4656-96c9-d1fad7e67308 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:08.018026 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73214993-db25-4656-96c9-d1fad7e67308 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:08.019486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73214993-db25-4656-96c9-d1fad7e67308 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:08.032244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 497/988] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:15:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:08.038079 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b1032-fe07-4c15-a73f-a1ac53dfb04e None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:08.127640 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:08.143150 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7b1032-fe07-4c15-a73f-a1ac53dfb04e None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:08.178324 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfdbd4a-1a50-45d3-a15d-a2fc077e3e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:08.178982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdbd4a-1a50-45d3-a15d-a2fc077e3e61 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:08.179292 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdbd4a-1a50-45d3-a15d-a2fc077e3e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:08.179563 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfdbd4a-1a50-45d3-a15d-a2fc077e3e61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:08.180392 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfdbd4a-1a50-45d3-a15d-a2fc077e3e61 None admin] RBAC: Adding query filter params (role.id=0b2dccc298c3428f91cea20447956c6a, scope.project.id=50902e0539434c82b505dc5009aeaf85, user.id=66ad8defb9b6453c96f1a8a7343695ba) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:08.180703 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfdbd4a-1a50-45d3-a15d-a2fc077e3e61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:08.182170 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfdbd4a-1a50-45d3-a15d-a2fc077e3e61 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:08.188346 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 492/989] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:15:08 2022] GET /identity/v3/role_assignments?user.id=66ad8defb9b6453c96f1a8a7343695ba&scope.project.id=50902e0539434c82b505dc5009aeaf85&role.id=0b2dccc298c3428f91cea20447956c6a => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:09.076333 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e77a5-a21e-4a23-af65-83fe6a999170 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:09.076879 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e77a5-a21e-4a23-af65-83fe6a999170 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:09.076879 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055e77a5-a21e-4a23-af65-83fe6a999170 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:09.079009 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 498/990] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:15:09.084295 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746766a-ed3d-4c58-99ef-93667beec66d None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:09.084726 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746766a-ed3d-4c58-99ef-93667beec66d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:09.084893 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746766a-ed3d-4c58-99ef-93667beec66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:09.101749 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-c746766a-ed3d-4c58-99ef-93667beec66d None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:09.102922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-c746766a-ed3d-4c58-99ef-93667beec66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:09.169730 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 493/991] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:09 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:15:09.193015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e77a5-a21e-4a23-af65-83fe6a999170 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:09.296422 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055e77a5-a21e-4a23-af65-83fe6a999170 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:09.299628 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:09.332624 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4534ef3b-e3a1-4775-9a6d-6ed4f7066322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:09.333348 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534ef3b-e3a1-4775-9a6d-6ed4f7066322 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:09.333628 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534ef3b-e3a1-4775-9a6d-6ed4f7066322 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:09.333905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534ef3b-e3a1-4775-9a6d-6ed4f7066322 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:09.334576 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4534ef3b-e3a1-4775-9a6d-6ed4f7066322 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:09.336146 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4534ef3b-e3a1-4775-9a6d-6ed4f7066322 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:09.349447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-4534ef3b-e3a1-4775-9a6d-6ed4f7066322 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 10:15:09.349447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:09.349447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 10:15:09.349447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 10:15:09.349447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 10:15:09.349447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 10:15:09.349447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 10:15:09.349447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:15:09.349447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 10:15:09.349447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:09.349447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:15:09.349447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:09.349447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:09.349447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:09.349447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:09.349447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:09.349447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:09.350489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:09.350489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:09.350489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:09.350489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:09.350489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:09.350489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:09.350489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 10:15:09.350489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 10:15:09.350489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 10:15:09.350489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 10:15:09.350489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:09.350489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:09.350489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:09.350489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:09.350489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:09.350489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:09.351841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:09.351841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:09.351841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:09.351841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:09.351841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:09.351841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:09.351841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:09.351841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:09.351841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:09.351841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:09.351841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 10:15:09.351841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:15:09.351841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 10:15:09.351841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:09.355380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 499/992] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:15:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:15:09.356712 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746766a-ed3d-4c58-99ef-93667beec66d None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:09.365397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821aeef-82c7-40d0-9785-f904614b853e None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:09.365683 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821aeef-82c7-40d0-9785-f904614b853e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:09.365967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0821aeef-82c7-40d0-9785-f904614b853e None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:09.367385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 500/993] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:15:09.372253 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19593c5-9e05-47b5-b12b-86197cccdbe3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:09.372548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19593c5-9e05-47b5-b12b-86197cccdbe3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:09.372834 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19593c5-9e05-47b5-b12b-86197cccdbe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:09.390742 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d19593c5-9e05-47b5-b12b-86197cccdbe3 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:09.392340 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d19593c5-9e05-47b5-b12b-86197cccdbe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:09.439805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:09.460005 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746766a-ed3d-4c58-99ef-93667beec66d None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:09.463453 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 501/994] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:09 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 10:15:09.483151 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19593c5-9e05-47b5-b12b-86197cccdbe3 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:09.501182 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975b46d-6447-4d0b-8bd3-8838f07068d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:09.501864 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975b46d-6447-4d0b-8bd3-8838f07068d1 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:09.503796 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975b46d-6447-4d0b-8bd3-8838f07068d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:09.503796 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5975b46d-6447-4d0b-8bd3-8838f07068d1 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:09.504522 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975b46d-6447-4d0b-8bd3-8838f07068d1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:09.504522 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975b46d-6447-4d0b-8bd3-8838f07068d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:09.505541 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5975b46d-6447-4d0b-8bd3-8838f07068d1 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:09.510592 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 494/995] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jun 30 10:15:09 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:09.516095 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975b46d-6447-4d0b-8bd3-8838f07068d1 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:09.601041 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:09.606491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19593c5-9e05-47b5-b12b-86197cccdbe3 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:09.626928 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5975b46d-6447-4d0b-8bd3-8838f07068d1 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:09.649302 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412a18e-d055-44dd-ab46-283c6f57c7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:09.650105 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412a18e-d055-44dd-ab46-283c6f57c7e7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:09.650602 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412a18e-d055-44dd-ab46-283c6f57c7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:09.650735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412a18e-d055-44dd-ab46-283c6f57c7e7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:09.651321 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412a18e-d055-44dd-ab46-283c6f57c7e7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:09.652694 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412a18e-d055-44dd-ab46-283c6f57c7e7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:09.662158 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb5649-450d-4db3-89fc-ca19828caf44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:09.662823 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb5649-450d-4db3-89fc-ca19828caf44 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:09.663109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb5649-450d-4db3-89fc-ca19828caf44 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:09.663420 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccb5649-450d-4db3-89fc-ca19828caf44 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:09.664131 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccb5649-450d-4db3-89fc-ca19828caf44 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:09.664440 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccb5649-450d-4db3-89fc-ca19828caf44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:09.665730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-4412a18e-d055-44dd-ab46-283c6f57c7e7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 10:15:09.665730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:09.665730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 10:15:09.665730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 10:15:09.665730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 10:15:09.665730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 10:15:09.665730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 10:15:09.665730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:15:09.665730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 10:15:09.665730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:09.665730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:15:09.665730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:09.665730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:09.665730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:09.665730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:09.665730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:09.665730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:09.666695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:09.666695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:09.666695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:09.666695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:09.666695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:09.666695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:09.666695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 10:15:09.666695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 10:15:09.666695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 10:15:09.666695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 10:15:09.666695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:09.666695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:09.666695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:09.666695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:09.666695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:09.666695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:09.667587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:09.667587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:09.667587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:09.667587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:09.667587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:09.667587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:09.667587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:09.667587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:09.667587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:09.667587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:09.667587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 10:15:09.667587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:15:09.667587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 30 10:15:09.667587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:09.668363 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccb5649-450d-4db3-89fc-ca19828caf44 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:09.668434 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 502/996] 10.222.0.47 () {56 vars in 1126 bytes} [Thu Jun 30 10:15:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:15:09.678097 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 495/997] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jun 30 10:15:09 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:09.678272 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412a18e-d055-44dd-ab46-283c6f57c7e7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:09.678785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb5649-450d-4db3-89fc-ca19828caf44 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:09.689824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:09.783145 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412a18e-d055-44dd-ab46-283c6f57c7e7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:09.789742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccb5649-450d-4db3-89fc-ca19828caf44 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:09.818059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3c0aa-46fe-4134-8e99-8b0b0494051c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:09.818739 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3c0aa-46fe-4134-8e99-8b0b0494051c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:09.819024 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3c0aa-46fe-4134-8e99-8b0b0494051c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:09.819327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3c0aa-46fe-4134-8e99-8b0b0494051c None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:09.820056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb3c0aa-46fe-4134-8e99-8b0b0494051c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:09.820357 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb3c0aa-46fe-4134-8e99-8b0b0494051c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:09.821732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb3c0aa-46fe-4134-8e99-8b0b0494051c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:09.826614 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 503/998] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jun 30 10:15:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:09.833287 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3c0aa-46fe-4134-8e99-8b0b0494051c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:09.833418 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99728cdf-5cb1-4138-8c0c-829cadc592c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:09.834082 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99728cdf-5cb1-4138-8c0c-829cadc592c5 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:09.834341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99728cdf-5cb1-4138-8c0c-829cadc592c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:09.834647 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99728cdf-5cb1-4138-8c0c-829cadc592c5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:09.835487 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99728cdf-5cb1-4138-8c0c-829cadc592c5 None admin] RBAC: Adding query filter params (interface=public, service_id=b2da2548f0fb464ea5f5c3a764e7e0ff, region_id=RegionOne) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:09.835927 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99728cdf-5cb1-4138-8c0c-829cadc592c5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:09.837301 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99728cdf-5cb1-4138-8c0c-829cadc592c5 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:09.842608 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 496/999] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Jun 30 10:15:09 2022] GET /identity/v3/endpoints?service_id=b2da2548f0fb464ea5f5c3a764e7e0ff&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:09.848690 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99728cdf-5cb1-4138-8c0c-829cadc592c5 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:09.866509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:09.927352 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:09.949445 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3c0aa-46fe-4134-8e99-8b0b0494051c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:09.958029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99728cdf-5cb1-4138-8c0c-829cadc592c5 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:09.989057 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba2576-e2ea-44e3-98ce-de04af030a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:09.989726 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba2576-e2ea-44e3-98ce-de04af030a4c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:09.990014 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba2576-e2ea-44e3-98ce-de04af030a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:09.990328 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba2576-e2ea-44e3-98ce-de04af030a4c None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:09.991022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba2576-e2ea-44e3-98ce-de04af030a4c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:09.991357 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba2576-e2ea-44e3-98ce-de04af030a4c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:09.992730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba2576-e2ea-44e3-98ce-de04af030a4c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:09.996322 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27a42ef-abc8-4581-ae65-16073a2518f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:09.996995 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a42ef-abc8-4581-ae65-16073a2518f3 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:09.997285 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a42ef-abc8-4581-ae65-16073a2518f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:09.997725 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27a42ef-abc8-4581-ae65-16073a2518f3 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:09.997827 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 504/1000] 10.222.0.47 () {56 vars in 1122 bytes} [Thu Jun 30 10:15:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:09.998382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a42ef-abc8-4581-ae65-16073a2518f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:10.000550 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27a42ef-abc8-4581-ae65-16073a2518f3 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:10.003683 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba2576-e2ea-44e3-98ce-de04af030a4c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:10.008214 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 497/1001] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jun 30 10:15:09 2022] GET /identity/v3/services => generated 1727 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:15:10.110030 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:10.123051 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba2576-e2ea-44e3-98ce-de04af030a4c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:10.158466 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758b65e-f8c9-485d-9ff4-7298e28de7b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:10.160434 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758b65e-f8c9-485d-9ff4-7298e28de7b9 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:10.160434 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758b65e-f8c9-485d-9ff4-7298e28de7b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:10.160434 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758b65e-f8c9-485d-9ff4-7298e28de7b9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:10.160672 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e758b65e-f8c9-485d-9ff4-7298e28de7b9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:10.161904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e758b65e-f8c9-485d-9ff4-7298e28de7b9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:10.280986 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 505/1002] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jun 30 10:15:10 2022] POST /identity/v3/endpoints => generated 325 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:15:10.282283 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee43a8-d70d-4256-866f-2d1a026ab8f7 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:10.282728 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee43a8-d70d-4256-866f-2d1a026ab8f7 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:10.282922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee43a8-d70d-4256-866f-2d1a026ab8f7 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:10.284569 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 498/1003] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:15:10.290258 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cbe06-9098-47ee-9f50-1a38d6cfa173 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:10.295179 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cbe06-9098-47ee-9f50-1a38d6cfa173 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:10.295179 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060cbe06-9098-47ee-9f50-1a38d6cfa173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:10.309515 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-060cbe06-9098-47ee-9f50-1a38d6cfa173 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:10.310752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-060cbe06-9098-47ee-9f50-1a38d6cfa173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:10.395400 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 506/1004] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:10 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:15:10.416469 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee43a8-d70d-4256-866f-2d1a026ab8f7 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:10.527542 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee43a8-d70d-4256-866f-2d1a026ab8f7 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:10.563918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39284032-d150-4b9f-8ca1-4cd9ed633d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:10.566973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39284032-d150-4b9f-8ca1-4cd9ed633d5f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:10.566973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39284032-d150-4b9f-8ca1-4cd9ed633d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:10.566973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39284032-d150-4b9f-8ca1-4cd9ed633d5f None admin] PATH_INFO: `/v3/services/image` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:10.566973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39284032-d150-4b9f-8ca1-4cd9ed633d5f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:10.568146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39284032-d150-4b9f-8ca1-4cd9ed633d5f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:10.570848 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:10.582951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-39284032-d150-4b9f-8ca1-4cd9ed633d5f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 10:15:10.582951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:10.582951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 10:15:10.582951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 10:15:10.582951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 10:15:10.582951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 10:15:10.582951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 10:15:10.582951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:15:10.582951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 10:15:10.582951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:10.582951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:15:10.582951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:10.582951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:10.582951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:10.582951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:10.582951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:10.582951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:10.593042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:10.593042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:10.593042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:10.593042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:10.593042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:10.593042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:10.593042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 10:15:10.593042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 10:15:10.593042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 10:15:10.593042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 10:15:10.593042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:10.593042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:10.593042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:10.593042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:10.593042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:10.593042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:10.594391 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:10.594391 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:10.594391 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:10.594391 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:10.594391 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:10.594391 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:10.594391 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:10.594391 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:10.594391 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:10.594391 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:10.594391 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 10:15:10.594391 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:15:10.594391 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 10:15:10.594391 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:10.594391 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 499/1005] 10.222.0.47 () {56 vars in 1105 bytes} [Thu Jun 30 10:15:10 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:15:10.595246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060cbe06-9098-47ee-9f50-1a38d6cfa173 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:10.676023 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:10.699240 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060cbe06-9098-47ee-9f50-1a38d6cfa173 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:10.734592 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831197bb-8ff9-4cf9-a6ab-a1af4e1b2bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:10.735383 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831197bb-8ff9-4cf9-a6ab-a1af4e1b2bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:10.735670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831197bb-8ff9-4cf9-a6ab-a1af4e1b2bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:10.735943 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831197bb-8ff9-4cf9-a6ab-a1af4e1b2bb0 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:10.736648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831197bb-8ff9-4cf9-a6ab-a1af4e1b2bb0 None admin] RBAC: Adding query filter params (name=image) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:10.736955 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831197bb-8ff9-4cf9-a6ab-a1af4e1b2bb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:10.738330 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831197bb-8ff9-4cf9-a6ab-a1af4e1b2bb0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:10.748668 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 507/1006] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:15:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:10.749818 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39284032-d150-4b9f-8ca1-4cd9ed633d5f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:10.852323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39284032-d150-4b9f-8ca1-4cd9ed633d5f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:10.884860 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f7b88-a946-4044-8541-d581dceb1869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:10.885529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f7b88-a946-4044-8541-d581dceb1869 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:10.885812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f7b88-a946-4044-8541-d581dceb1869 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:10.886090 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f7b88-a946-4044-8541-d581dceb1869 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:10.886794 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f7b88-a946-4044-8541-d581dceb1869 None admin] RBAC: Adding query filter params (type=image) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:10.887102 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f7b88-a946-4044-8541-d581dceb1869 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:10.888496 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f7b88-a946-4044-8541-d581dceb1869 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:10.899300 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 500/1007] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:15:10 2022] GET /identity/v3/services?type=image => generated 122 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:12.078798 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6083e08-4b0a-448c-b6b5-1c087fd6f61b None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:12.079539 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6083e08-4b0a-448c-b6b5-1c087fd6f61b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:12.079539 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6083e08-4b0a-448c-b6b5-1c087fd6f61b None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:12.080792 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 508/1008] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:15:12.085903 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ae74f-2c58-45c2-80a4-c70896b13cfe None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:12.086300 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ae74f-2c58-45c2-80a4-c70896b13cfe None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:12.086493 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94ae74f-2c58-45c2-80a4-c70896b13cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:12.103963 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-e94ae74f-2c58-45c2-80a4-c70896b13cfe None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:12.105147 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-e94ae74f-2c58-45c2-80a4-c70896b13cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:12.165552 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:12.175040 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 501/1009] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:12 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:15:12.198164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6083e08-4b0a-448c-b6b5-1c087fd6f61b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:12.317915 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6083e08-4b0a-448c-b6b5-1c087fd6f61b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:12.351963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9031fe-a5d0-4a5c-a899-a60d31f15454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:12.352676 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9031fe-a5d0-4a5c-a899-a60d31f15454 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:12.352957 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9031fe-a5d0-4a5c-a899-a60d31f15454 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:12.353230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9031fe-a5d0-4a5c-a899-a60d31f15454 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:12.353880 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9031fe-a5d0-4a5c-a899-a60d31f15454 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:12.355246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9031fe-a5d0-4a5c-a899-a60d31f15454 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:12.363109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ca0e8-52df-46c6-9174-2dc2b9fabdb3 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:12.363434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ca0e8-52df-46c6-9174-2dc2b9fabdb3 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:12.363716 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ca0e8-52df-46c6-9174-2dc2b9fabdb3 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:12.365217 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 502/1010] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:15:12.367056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-8f9031fe-a5d0-4a5c-a899-a60d31f15454 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 10:15:12.367056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:12.367056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 10:15:12.367056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 10:15:12.367056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 10:15:12.367056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 10:15:12.367056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 10:15:12.367056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:15:12.367056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 10:15:12.367056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:12.367056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:15:12.367056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:12.367056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:12.367056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:12.367056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:12.367056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:12.367056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:12.368067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:12.368067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:12.368067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:12.368067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:12.368067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:12.368067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:12.368067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 10:15:12.368067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 10:15:12.368067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 10:15:12.368067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 10:15:12.368067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:12.368067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:12.368067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:12.368067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:12.368067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:12.368067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:12.368936 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:12.368936 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:12.368936 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:12.368936 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:12.368936 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:12.368936 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:12.368936 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:12.368936 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:12.368936 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:12.368936 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:12.368936 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 10:15:12.368936 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:15:12.368936 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 30 10:15:12.368936 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:12.370179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884a170-9743-4fdf-9830-44bf5a5f2df4 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:12.370414 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884a170-9743-4fdf-9830-44bf5a5f2df4 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:12.370690 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884a170-9743-4fdf-9830-44bf5a5f2df4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:12.371555 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 509/1011] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:15:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:15:12.373841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9031fe-a5d0-4a5c-a899-a60d31f15454 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:12.378220 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:12.391412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-9884a170-9743-4fdf-9830-44bf5a5f2df4 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:12.392838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-9884a170-9743-4fdf-9830-44bf5a5f2df4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:12.473391 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 503/1012] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:12 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:15:12.481166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9031fe-a5d0-4a5c-a899-a60d31f15454 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:12.496794 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884a170-9743-4fdf-9830-44bf5a5f2df4 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:12.523040 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8998756-ac83-4b22-b5db-82597f7b54b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:12.524007 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8998756-ac83-4b22-b5db-82597f7b54b7 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:12.524292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8998756-ac83-4b22-b5db-82597f7b54b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:12.524567 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8998756-ac83-4b22-b5db-82597f7b54b7 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:12.525431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8998756-ac83-4b22-b5db-82597f7b54b7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:12.526049 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8998756-ac83-4b22-b5db-82597f7b54b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:12.527915 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8998756-ac83-4b22-b5db-82597f7b54b7 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:12.534037 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 510/1013] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jun 30 10:15:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:12.539517 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8998756-ac83-4b22-b5db-82597f7b54b7 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:12.614490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884a170-9743-4fdf-9830-44bf5a5f2df4 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:12.614679 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:12.647414 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:12.656972 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8998756-ac83-4b22-b5db-82597f7b54b7 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:12.658899 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f4ffc-653b-4fa1-9050-a3ef290dd8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:12.659937 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f4ffc-653b-4fa1-9050-a3ef290dd8ac None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:12.660221 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f4ffc-653b-4fa1-9050-a3ef290dd8ac None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:12.660505 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036f4ffc-653b-4fa1-9050-a3ef290dd8ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:12.661693 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f4ffc-653b-4fa1-9050-a3ef290dd8ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:12.663130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036f4ffc-653b-4fa1-9050-a3ef290dd8ac None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:12.664991 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 504/1014] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:15:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:12.670458 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f4ffc-653b-4fa1-9050-a3ef290dd8ac None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:12.698503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa3f52-44a7-4d14-9cf0-1be39968d017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:12.699473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa3f52-44a7-4d14-9cf0-1be39968d017 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:12.699916 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa3f52-44a7-4d14-9cf0-1be39968d017 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:12.700194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afa3f52-44a7-4d14-9cf0-1be39968d017 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:12.701053 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa3f52-44a7-4d14-9cf0-1be39968d017 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:12.701226 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa3f52-44a7-4d14-9cf0-1be39968d017 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:12.702707 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afa3f52-44a7-4d14-9cf0-1be39968d017 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:12.708471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 511/1015] 10.222.0.47 () {56 vars in 1112 bytes} [Thu Jun 30 10:15:12 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:12.713891 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa3f52-44a7-4d14-9cf0-1be39968d017 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:12.781593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036f4ffc-653b-4fa1-9050-a3ef290dd8ac None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:12.821706 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afa3f52-44a7-4d14-9cf0-1be39968d017 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:12.822184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c95fa76-ec06-4bd9-8a45-c131d5370f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:12.823377 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95fa76-ec06-4bd9-8a45-c131d5370f9f None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:12.824721 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95fa76-ec06-4bd9-8a45-c131d5370f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:12.825200 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c95fa76-ec06-4bd9-8a45-c131d5370f9f None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:12.826102 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c95fa76-ec06-4bd9-8a45-c131d5370f9f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:12.827952 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c95fa76-ec06-4bd9-8a45-c131d5370f9f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:12.853123 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:12.858377 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46cf5a8-19c1-428d-afc2-b0c8cb0fd3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:12.859063 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46cf5a8-19c1-428d-afc2-b0c8cb0fd3f4 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:12.859544 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46cf5a8-19c1-428d-afc2-b0c8cb0fd3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:12.859837 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46cf5a8-19c1-428d-afc2-b0c8cb0fd3f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:12.860519 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46cf5a8-19c1-428d-afc2-b0c8cb0fd3f4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:12.861897 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46cf5a8-19c1-428d-afc2-b0c8cb0fd3f4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:12.954734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 505/1016] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jun 30 10:15:12 2022] POST /identity/v3/projects => generated 311 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:15:12.975785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e60cf5-dad7-470e-b12e-f7a88b1c6fa0 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:12.976081 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e60cf5-dad7-470e-b12e-f7a88b1c6fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:12.976349 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e60cf5-dad7-470e-b12e-f7a88b1c6fa0 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:12.977849 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 506/1017] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:15:12.989838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 512/1018] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jun 30 10:15:12 2022] POST /identity/v3/endpoints => generated 334 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:15:12.990056 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db4a14-9987-444a-9d12-782984f81ae1 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:12.990056 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db4a14-9987-444a-9d12-782984f81ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:12.990056 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db4a14-9987-444a-9d12-782984f81ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:13.003873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-f9db4a14-9987-444a-9d12-782984f81ae1 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:13.005169 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-f9db4a14-9987-444a-9d12-782984f81ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:13.053803 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:13.093678 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 507/1019] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:12 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 10:15:13.117652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46cf5a8-19c1-428d-afc2-b0c8cb0fd3f4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:13.226602 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46cf5a8-19c1-428d-afc2-b0c8cb0fd3f4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:13.269287 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66761a7-ffea-4071-a70a-d00483cc9ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:13.269967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66761a7-ffea-4071-a70a-d00483cc9ec4 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:13.270255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66761a7-ffea-4071-a70a-d00483cc9ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:13.270536 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66761a7-ffea-4071-a70a-d00483cc9ec4 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:13.271333 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66761a7-ffea-4071-a70a-d00483cc9ec4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:13.273013 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66761a7-ffea-4071-a70a-d00483cc9ec4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:13.367241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 513/1020] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jun 30 10:15:13 2022] POST /identity/v3/services => generated 242 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:15:15.076826 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e4baf-b88a-4a9c-a8ca-fb9e180f13dc None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:15.077357 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e4baf-b88a-4a9c-a8ca-fb9e180f13dc None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:15.077357 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e4baf-b88a-4a9c-a8ca-fb9e180f13dc None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:15.079026 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 508/1021] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:15:15.084083 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d56e0f-b703-4dd1-a156-297889f4ac6c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:15.084520 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d56e0f-b703-4dd1-a156-297889f4ac6c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:15.084814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d56e0f-b703-4dd1-a156-297889f4ac6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:15.101620 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-68d56e0f-b703-4dd1-a156-297889f4ac6c None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:15.102803 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-68d56e0f-b703-4dd1-a156-297889f4ac6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:15.148771 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:15.178583 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 514/1022] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:15 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:15:15.204168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e4baf-b88a-4a9c-a8ca-fb9e180f13dc None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:15.222945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8404b32-4ece-41ac-b043-bddd11694a94 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:15.223235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8404b32-4ece-41ac-b043-bddd11694a94 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:15.223555 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8404b32-4ece-41ac-b043-bddd11694a94 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:15.225066 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 515/1023] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:15:15.229766 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935bd5d-4685-41a1-a533-36765534259a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:15.230052 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935bd5d-4685-41a1-a533-36765534259a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:15.230329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c935bd5d-4685-41a1-a533-36765534259a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:15.248450 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c935bd5d-4685-41a1-a533-36765534259a None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:15.250003 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c935bd5d-4685-41a1-a533-36765534259a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:15.313425 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e4baf-b88a-4a9c-a8ca-fb9e180f13dc None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:15.327389 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 516/1024] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:15 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 10:15:15.349648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7caa7b8-9cb9-446f-bc36-804d1bcafa61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:15.350223 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935bd5d-4685-41a1-a533-36765534259a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:15.350446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7caa7b8-9cb9-446f-bc36-804d1bcafa61 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:15.350729 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7caa7b8-9cb9-446f-bc36-804d1bcafa61 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:15.351003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7caa7b8-9cb9-446f-bc36-804d1bcafa61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:15.352229 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7caa7b8-9cb9-446f-bc36-804d1bcafa61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:15.353675 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7caa7b8-9cb9-446f-bc36-804d1bcafa61 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:15.355313 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 509/1025] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:15:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:15.361346 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7caa7b8-9cb9-446f-bc36-804d1bcafa61 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:15.404657 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:15.458877 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c935bd5d-4685-41a1-a533-36765534259a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:15.461161 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:15.465452 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7caa7b8-9cb9-446f-bc36-804d1bcafa61 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:15.495156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777393db-744d-4cf4-8fe9-debefed09280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:15.495998 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777393db-744d-4cf4-8fe9-debefed09280 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:15.496279 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777393db-744d-4cf4-8fe9-debefed09280 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:15.496552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777393db-744d-4cf4-8fe9-debefed09280 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:15.501451 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935cac7b-855c-44c6-8ae9-e34e096ecdf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:15.502127 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935cac7b-855c-44c6-8ae9-e34e096ecdf5 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:15.502435 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935cac7b-855c-44c6-8ae9-e34e096ecdf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:15.502675 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935cac7b-855c-44c6-8ae9-e34e096ecdf5 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:15.503430 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935cac7b-855c-44c6-8ae9-e34e096ecdf5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:15.504858 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935cac7b-855c-44c6-8ae9-e34e096ecdf5 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:15.509327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777393db-744d-4cf4-8fe9-debefed09280 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:15:15.517502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-777393db-744d-4cf4-8fe9-debefed09280 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 10:15:15.517502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:15.517502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:15.517502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:15.517502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:15.517502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:15.517502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:15.517502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:15.517502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:15.517502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:15.517502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:15.517502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:15.517502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:15:15.517502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:15:15.517502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:15:15.517502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:15:15.520320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:15:15.520320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:15:15.520320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:15:15.520320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:15:15.520320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:15.520320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:15.520320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:15.520320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:15.520320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:15.520320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:15.520320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:15.520320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:15.520320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:15.520320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:15.520320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:15.520320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:15.521178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:15.521178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:15.521178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:15.521178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:15.521178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:15:15.521178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:15:15.521178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:15:15.521178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:15:15.521178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:15:15.521178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:15:15.521178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 10:15:15.521178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:15.521178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 517/1026] 10.222.0.47 () {56 vars in 1120 bytes} [Thu Jun 30 10:15:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:15:15.521178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b094b16-49ab-4e6b-a38b-5a80cdff7e25 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:15.523554 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b094b16-49ab-4e6b-a38b-5a80cdff7e25 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:15.523554 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b094b16-49ab-4e6b-a38b-5a80cdff7e25 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:15.523554 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 518/1027] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:15:15.525106 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b094b16-49ab-4e6b-a38b-5a80cdff7e25 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:15.605598 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 510/1028] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jun 30 10:15:15 2022] POST /identity/v3/users => generated 285 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:15:15.606814 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e491a8-8c2a-4c9e-83a6-b7f9bd13c969 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:15.607105 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e491a8-8c2a-4c9e-83a6-b7f9bd13c969 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:15.608581 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e491a8-8c2a-4c9e-83a6-b7f9bd13c969 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:15.632751 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b094b16-49ab-4e6b-a38b-5a80cdff7e25 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:15.636062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-89e491a8-8c2a-4c9e-83a6-b7f9bd13c969 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:15.637329 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-89e491a8-8c2a-4c9e-83a6-b7f9bd13c969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:15.674166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd0a108-3fd2-40bc-9494-09b97f6f220a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:15.675148 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0a108-3fd2-40bc-9494-09b97f6f220a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:15.675457 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0a108-3fd2-40bc-9494-09b97f6f220a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:15.675762 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd0a108-3fd2-40bc-9494-09b97f6f220a None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:15.676707 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd0a108-3fd2-40bc-9494-09b97f6f220a None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:15.677026 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd0a108-3fd2-40bc-9494-09b97f6f220a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:15.678458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd0a108-3fd2-40bc-9494-09b97f6f220a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:15.691316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 519/1029] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Jun 30 10:15:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:15.696920 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd0a108-3fd2-40bc-9494-09b97f6f220a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:15.722542 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 511/1030] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:15 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:15:15.728078 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:15.746521 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e491a8-8c2a-4c9e-83a6-b7f9bd13c969 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:15.775897 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:15.806868 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd0a108-3fd2-40bc-9494-09b97f6f220a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:15.846964 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b04ae-1d2d-4cb9-a89e-fda90773f6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:15.847800 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b04ae-1d2d-4cb9-a89e-fda90773f6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:15.848082 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b04ae-1d2d-4cb9-a89e-fda90773f6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:15.848353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b04ae-1d2d-4cb9-a89e-fda90773f6f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:15.859111 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b04ae-1d2d-4cb9-a89e-fda90773f6f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:15.860564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b04ae-1d2d-4cb9-a89e-fda90773f6f6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:15.869793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e491a8-8c2a-4c9e-83a6-b7f9bd13c969 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:15.878606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-b07b04ae-1d2d-4cb9-a89e-fda90773f6f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:15:15.878606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:15.878606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:15.878606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:15.878606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:15.878606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:15.878606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:15.878606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:15.878606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:15.878606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:15.878606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:15.878606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:15.878606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:15:15.878606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:15:15.878606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:15:15.879748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:15:15.879748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:15.879748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:15.879748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:15.879748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:15.879748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:15.879748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:15.879748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:15.879748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:15.879748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:15.879748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:15.879748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:15.879748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:15.879748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:15.879748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:15.879748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:15.879748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:15.885506 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:15:15.885506 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:15:15.885506 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:15:15.885506 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:15.885506 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 520/1031] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:15:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:15:15.886030 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b04ae-1d2d-4cb9-a89e-fda90773f6f6 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:15.907818 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e618c-0f3b-4016-a05f-2c249e4e3124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:15.908611 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e618c-0f3b-4016-a05f-2c249e4e3124 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:15.908896 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e618c-0f3b-4016-a05f-2c249e4e3124 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:15.909171 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e618c-0f3b-4016-a05f-2c249e4e3124 None admin] PATH_INFO: `/v3/services/image` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:15.909827 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e618c-0f3b-4016-a05f-2c249e4e3124 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:15.911196 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e618c-0f3b-4016-a05f-2c249e4e3124 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:15.923816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-8e9e618c-0f3b-4016-a05f-2c249e4e3124 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 10:15:15.923816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:15.923816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 10:15:15.923816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 10:15:15.923816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 10:15:15.923816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 10:15:15.923816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 10:15:15.923816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:15:15.923816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 10:15:15.923816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:15.923816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:15:15.923816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:15.923816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:15.923816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:15.923816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:15.923816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:15.923816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:15.925155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:15.925155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:15.925155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:15.925155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:15.925155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:15.925155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:15.925155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 10:15:15.925155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 10:15:15.925155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 10:15:15.925155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 10:15:15.925155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:15.925155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:15.925155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:15.925155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:15.925155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:15.925155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:15.926197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:15.926197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:15.926197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:15.926197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:15.926197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:15.926197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:15.926197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:15.926197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:15.926197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:15.926197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:15.926197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 10:15:15.926197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:15:15.926197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 10:15:15.926197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:15.926197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 512/1032] 10.222.0.47 () {56 vars in 1105 bytes} [Thu Jun 30 10:15:15 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:15:15.931016 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e618c-0f3b-4016-a05f-2c249e4e3124 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:15.983778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:16.006167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b04ae-1d2d-4cb9-a89e-fda90773f6f6 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:16.024794 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:16.038247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e618c-0f3b-4016-a05f-2c249e4e3124 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:16.043217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4102ce-0572-4e5e-9b07-5436fbeeb55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:16.043961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4102ce-0572-4e5e-9b07-5436fbeeb55a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:16.044217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4102ce-0572-4e5e-9b07-5436fbeeb55a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:16.044498 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4102ce-0572-4e5e-9b07-5436fbeeb55a None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:16.045237 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4102ce-0572-4e5e-9b07-5436fbeeb55a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:16.045551 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4102ce-0572-4e5e-9b07-5436fbeeb55a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:16.047290 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4102ce-0572-4e5e-9b07-5436fbeeb55a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:16.058842 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 521/1033] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:16.064780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4102ce-0572-4e5e-9b07-5436fbeeb55a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:16.079183 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594abcf-0593-4309-9f27-118b5351af6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:16.079901 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594abcf-0593-4309-9f27-118b5351af6a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:16.080190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594abcf-0593-4309-9f27-118b5351af6a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:16.080465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c594abcf-0593-4309-9f27-118b5351af6a None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:16.081316 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594abcf-0593-4309-9f27-118b5351af6a None admin] RBAC: Adding query filter params (name=image) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:16.081747 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594abcf-0593-4309-9f27-118b5351af6a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:16.083122 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c594abcf-0593-4309-9f27-118b5351af6a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:16.092968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 513/1034] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:15:15 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:16.094550 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594abcf-0593-4309-9f27-118b5351af6a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:16.170926 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4102ce-0572-4e5e-9b07-5436fbeeb55a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:16.199594 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c594abcf-0593-4309-9f27-118b5351af6a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:16.210243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01618b4c-b4a9-4a9c-9844-0a06a70df365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:16.211038 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01618b4c-b4a9-4a9c-9844-0a06a70df365 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:16.211359 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01618b4c-b4a9-4a9c-9844-0a06a70df365 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:16.211647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01618b4c-b4a9-4a9c-9844-0a06a70df365 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:16.220274 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01618b4c-b4a9-4a9c-9844-0a06a70df365 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:16.221726 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01618b4c-b4a9-4a9c-9844-0a06a70df365 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:16.223587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:16.237015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-01618b4c-b4a9-4a9c-9844-0a06a70df365 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 10:15:16.237015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:16.237015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:16.237015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:16.237015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:16.237015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:16.237015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:16.237015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:16.237015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:16.237015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:16.237015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:16.237015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:16.237015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:15:16.237015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:15:16.237015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:15:16.237015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:15:16.238176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:16.238176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:16.238176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:15:16.238176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:15:16.238176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:15:16.238176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:15:16.238176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:16.238176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:16.238176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:16.238176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:16.238176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:16.238176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:16.238176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:16.238176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:16.238176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:16.238176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:16.238176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:16.238176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:16.239631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:16.239631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:16.239631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:15:16.239631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:15:16.239631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:15:16.239631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:15:16.239631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:15:16.239631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:15:16.239631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 10:15:16.239631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:16.239631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 522/1035] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jun 30 10:15:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:15:16.242100 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d31d9e-e768-465f-8273-df5a36c2457b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:16.242838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d31d9e-e768-465f-8273-df5a36c2457b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:16.243239 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d31d9e-e768-465f-8273-df5a36c2457b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:16.243552 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d31d9e-e768-465f-8273-df5a36c2457b None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:16.244069 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01618b4c-b4a9-4a9c-9844-0a06a70df365 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:16.244333 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d31d9e-e768-465f-8273-df5a36c2457b None admin] RBAC: Adding query filter params (type=image) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:16.244644 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d31d9e-e768-465f-8273-df5a36c2457b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:16.246006 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d31d9e-e768-465f-8273-df5a36c2457b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:16.258248 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 514/1036] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:15:16 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:16.258904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d31d9e-e768-465f-8273-df5a36c2457b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:16.277867 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:16.357306 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01618b4c-b4a9-4a9c-9844-0a06a70df365 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:16.364736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d31d9e-e768-465f-8273-df5a36c2457b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:16.398176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448327c-35c9-4e28-bef2-fee4381135f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:16.398879 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448327c-35c9-4e28-bef2-fee4381135f4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:16.399244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448327c-35c9-4e28-bef2-fee4381135f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:16.399569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0448327c-35c9-4e28-bef2-fee4381135f4 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:16.400405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0448327c-35c9-4e28-bef2-fee4381135f4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:16.400715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0448327c-35c9-4e28-bef2-fee4381135f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:16.402136 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0448327c-35c9-4e28-bef2-fee4381135f4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:16.402717 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49662c-3959-4ab2-930c-ef59ad91aa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:16.403472 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49662c-3959-4ab2-930c-ef59ad91aa07 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:16.403824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49662c-3959-4ab2-930c-ef59ad91aa07 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:16.404171 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49662c-3959-4ab2-930c-ef59ad91aa07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:16.405034 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e49662c-3959-4ab2-930c-ef59ad91aa07 None admin] RBAC: Adding query filter params (interface=public, service_id=ce22fdb9ac344dedbf1e5ea5de29abaf, region_id=RegionOne) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:16.405417 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e49662c-3959-4ab2-930c-ef59ad91aa07 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:16.406870 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e49662c-3959-4ab2-930c-ef59ad91aa07 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:16.412783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 515/1037] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Jun 30 10:15:16 2022] GET /identity/v3/endpoints?service_id=ce22fdb9ac344dedbf1e5ea5de29abaf&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:16.417501 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49662c-3959-4ab2-930c-ef59ad91aa07 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:16.429604 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 523/1038] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jun 30 10:15:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:16.430530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448327c-35c9-4e28-bef2-fee4381135f4 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:16.492268 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:16.511757 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:16.536414 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49662c-3959-4ab2-930c-ef59ad91aa07 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:16.538550 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0448327c-35c9-4e28-bef2-fee4381135f4 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:16.574020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f597631f-1674-4479-ab97-2eb5b8b764fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:16.574688 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f597631f-1674-4479-ab97-2eb5b8b764fc None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:16.574975 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f597631f-1674-4479-ab97-2eb5b8b764fc None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:16.575070 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb6eb9d-5a8d-467f-b169-103fb89cd801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:16.575256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f597631f-1674-4479-ab97-2eb5b8b764fc None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:16.575709 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6eb9d-5a8d-467f-b169-103fb89cd801 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:16.575987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f597631f-1674-4479-ab97-2eb5b8b764fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:16.576079 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6eb9d-5a8d-467f-b169-103fb89cd801 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:16.576261 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb6eb9d-5a8d-467f-b169-103fb89cd801 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:16.577360 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f597631f-1674-4479-ab97-2eb5b8b764fc None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:16.584759 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 516/1039] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jun 30 10:15:16 2022] GET /identity/v3/services => generated 1957 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:15:16.655017 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb6eb9d-5a8d-467f-b169-103fb89cd801 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:16.656599 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb6eb9d-5a8d-467f-b169-103fb89cd801 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:16.717720 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:16.766124 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 524/1040] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:15:16 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 10:15:16.771727 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f597631f-1674-4479-ab97-2eb5b8b764fc None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:16.876844 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f597631f-1674-4479-ab97-2eb5b8b764fc None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:16.910285 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796225cb-4871-4e85-a8df-362ee4f1a15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:16.911084 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796225cb-4871-4e85-a8df-362ee4f1a15e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:16.911396 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796225cb-4871-4e85-a8df-362ee4f1a15e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:16.911675 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796225cb-4871-4e85-a8df-362ee4f1a15e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:16.922853 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796225cb-4871-4e85-a8df-362ee4f1a15e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:16.924327 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796225cb-4871-4e85-a8df-362ee4f1a15e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:16.926872 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:16.942563 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-796225cb-4871-4e85-a8df-362ee4f1a15e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:15:16.942563 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:16.942563 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:16.942563 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:16.942563 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:16.942563 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:16.942563 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:16.942563 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:16.942563 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:16.942563 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:16.942563 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:16.942563 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:16.942563 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:15:16.942563 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:15:16.942563 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:15:16.943507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:15:16.943507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:16.943507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:16.943507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:16.943507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:16.943507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:16.943507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:16.943507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:16.943507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:16.943507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:16.943507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:16.943507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:16.943507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:16.943507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:16.943507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:16.943507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:16.943507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:16.944390 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:15:16.944390 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:15:16.944390 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:15:16.944390 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:16.947385 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 517/1041] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:15:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:15:16.949762 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb6eb9d-5a8d-467f-b169-103fb89cd801 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:17.055318 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb6eb9d-5a8d-467f-b169-103fb89cd801 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:17.089357 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7602db5-de66-413b-9380-e2df486c285b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:17.090025 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7602db5-de66-413b-9380-e2df486c285b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:17.090306 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7602db5-de66-413b-9380-e2df486c285b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:17.090577 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7602db5-de66-413b-9380-e2df486c285b None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:17.091310 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7602db5-de66-413b-9380-e2df486c285b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:17.091618 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7602db5-de66-413b-9380-e2df486c285b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:17.092975 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7602db5-de66-413b-9380-e2df486c285b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:17.110132 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 525/1042] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:15:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:17.111956 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796225cb-4871-4e85-a8df-362ee4f1a15e None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:17.215868 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796225cb-4871-4e85-a8df-362ee4f1a15e None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:17.250378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051efadc-923b-41a5-b735-7f044f79c8ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:17.251185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051efadc-923b-41a5-b735-7f044f79c8ec None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:17.251501 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051efadc-923b-41a5-b735-7f044f79c8ec None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:17.251779 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051efadc-923b-41a5-b735-7f044f79c8ec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:17.258516 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051efadc-923b-41a5-b735-7f044f79c8ec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:17.259983 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051efadc-923b-41a5-b735-7f044f79c8ec None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:17.274446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-051efadc-923b-41a5-b735-7f044f79c8ec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:15:17.274446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:17.274446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:17.274446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:17.274446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:17.274446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:17.274446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:17.274446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:17.274446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:17.274446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:17.274446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:17.274446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:17.274446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:15:17.274446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:15:17.274446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:15:17.275634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:15:17.275634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:17.275634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:17.275634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:17.275634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:17.275634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:17.275634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:17.275634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:17.275634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:17.275634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:17.275634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:17.275634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:17.275634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:17.275634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:17.275634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:17.275634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:17.275634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:17.276611 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:15:17.276611 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:15:17.276611 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:15:17.276611 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:15:17.276611 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:15:17.276611 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:15:17.276611 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:15:17.276611 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:17.279326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 518/1043] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:15:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:15:17.282405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7602db5-de66-413b-9380-e2df486c285b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:17.302666 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:17.386529 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7602db5-de66-413b-9380-e2df486c285b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:17.422138 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6d512-c6c9-47db-9951-6654f22a3450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:17.422990 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6d512-c6c9-47db-9951-6654f22a3450 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:17.423415 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6d512-c6c9-47db-9951-6654f22a3450 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:17.423814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6d512-c6c9-47db-9951-6654f22a3450 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:17.424706 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6d512-c6c9-47db-9951-6654f22a3450 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:17.425120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6d512-c6c9-47db-9951-6654f22a3450 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:17.426650 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6d512-c6c9-47db-9951-6654f22a3450 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:17.443587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 526/1044] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:15:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:17.445077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051efadc-923b-41a5-b735-7f044f79c8ec None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:17.506502 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:17.550571 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051efadc-923b-41a5-b735-7f044f79c8ec None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:17.585324 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a3415-5972-4de9-997d-0ecf5b23fb98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:17.585985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a3415-5972-4de9-997d-0ecf5b23fb98 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:17.586269 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a3415-5972-4de9-997d-0ecf5b23fb98 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:17.586547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a3415-5972-4de9-997d-0ecf5b23fb98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:17.587405 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594a3415-5972-4de9-997d-0ecf5b23fb98 None admin] RBAC: Adding query filter params (role.id=0b2dccc298c3428f91cea20447956c6a, scope.project.id=50902e0539434c82b505dc5009aeaf85, user.id=71f6d7715e6f49ffb9fdf6e1307bbd51) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:17.587723 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594a3415-5972-4de9-997d-0ecf5b23fb98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:17.589154 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594a3415-5972-4de9-997d-0ecf5b23fb98 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:17.595344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 519/1045] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:15:17 2022] GET /identity/v3/role_assignments?user.id=71f6d7715e6f49ffb9fdf6e1307bbd51&scope.project.id=50902e0539434c82b505dc5009aeaf85&role.id=0b2dccc298c3428f91cea20447956c6a => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:17.724592 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4d0e5-4512-4ca1-b993-c66684206c73 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:17.724932 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4d0e5-4512-4ca1-b993-c66684206c73 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:17.725149 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b4d0e5-4512-4ca1-b993-c66684206c73 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:17.726750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 527/1046] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:15:17.732255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d527b-85eb-4eec-b7d4-ea6b8df6a967 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:17.732546 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d527b-85eb-4eec-b7d4-ea6b8df6a967 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:17.732822 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d527b-85eb-4eec-b7d4-ea6b8df6a967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:17.751337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-df2d527b-85eb-4eec-b7d4-ea6b8df6a967 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:17.752786 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-df2d527b-85eb-4eec-b7d4-ea6b8df6a967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:17.822740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 520/1047] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:17 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:15:17.846455 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4d0e5-4512-4ca1-b993-c66684206c73 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:17.928414 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:17.952033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b4d0e5-4512-4ca1-b993-c66684206c73 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:17.985736 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8170629-eec2-458f-b84c-827d74ee7f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:17.986463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8170629-eec2-458f-b84c-827d74ee7f59 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:17.986746 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8170629-eec2-458f-b84c-827d74ee7f59 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:17.987022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8170629-eec2-458f-b84c-827d74ee7f59 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:17.997204 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8170629-eec2-458f-b84c-827d74ee7f59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:15:18.005329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-c8170629-eec2-458f-b84c-827d74ee7f59 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:15:18.005329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:18.005329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:18.005329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:18.005329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:18.005329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:18.005329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:18.005329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:18.005329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:18.005329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:18.005329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:18.005329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:18.005329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:15:18.005329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:15:18.005329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:15:18.005329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:15:18.006293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:15:18.006293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:15:18.006293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:15:18.006293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:15:18.006293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:18.006293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:18.006293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:18.006293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:18.006293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:18.006293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:18.006293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:18.006293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:18.006293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:18.006293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:18.006293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:18.006293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:18.007215 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:18.007215 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:18.007215 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:18.007215 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:18.007215 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:15:18.007215 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:15:18.007215 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:15:18.007215 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:15:18.007215 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:15:18.007215 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:15:18.007215 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:15:18.007215 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:18.007215 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 528/1048] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:15:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:15:18.014652 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d527b-85eb-4eec-b7d4-ea6b8df6a967 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:18.069357 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:18.118995 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d527b-85eb-4eec-b7d4-ea6b8df6a967 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:18.154704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e1b7c-9142-43d9-a636-bc28a2243884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:18.155409 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e1b7c-9142-43d9-a636-bc28a2243884 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:18.155696 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e1b7c-9142-43d9-a636-bc28a2243884 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:18.156002 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6e1b7c-9142-43d9-a636-bc28a2243884 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:18.156827 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e1b7c-9142-43d9-a636-bc28a2243884 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:18.157144 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e1b7c-9142-43d9-a636-bc28a2243884 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:18.158512 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6e1b7c-9142-43d9-a636-bc28a2243884 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:18.166834 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 521/1049] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:15:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:18.172388 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8170629-eec2-458f-b84c-827d74ee7f59 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:18.274687 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8170629-eec2-458f-b84c-827d74ee7f59 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:18.308629 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43879bbe-f8b0-453b-b63b-f57487b812ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:18.309511 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43879bbe-f8b0-453b-b63b-f57487b812ee None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:18.309796 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43879bbe-f8b0-453b-b63b-f57487b812ee None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:18.310070 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43879bbe-f8b0-453b-b63b-f57487b812ee None admin] PATH_INFO: `/v3/users/ddbf7d7fd1bd4931bf22b3985fc0bdc6` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:18.313623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:18.325235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43879bbe-f8b0-453b-b63b-f57487b812ee None admin] RBAC: Authorizing `identity:get_user(user_id=ddbf7d7fd1bd4931bf22b3985fc0bdc6)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:18.326691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43879bbe-f8b0-453b-b63b-f57487b812ee None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:18.328708 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 529/1050] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:15:18 2022] GET /identity/v3/users/ddbf7d7fd1bd4931bf22b3985fc0bdc6 => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:18.334238 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e1b7c-9142-43d9-a636-bc28a2243884 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:18.434093 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6e1b7c-9142-43d9-a636-bc28a2243884 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:18.446608 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:18.467705 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2183da3-66ab-4bb9-a6bb-9d419d931ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:18.468901 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2183da3-66ab-4bb9-a6bb-9d419d931ced None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:18.469185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2183da3-66ab-4bb9-a6bb-9d419d931ced None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:18.469464 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2183da3-66ab-4bb9-a6bb-9d419d931ced None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:18.543097 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2183da3-66ab-4bb9-a6bb-9d419d931ced None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:18.544507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2183da3-66ab-4bb9-a6bb-9d419d931ced None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:18.645092 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73010435-cb6d-49e7-a051-6eb8942074ae None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:18.645437 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73010435-cb6d-49e7-a051-6eb8942074ae None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:18.645753 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73010435-cb6d-49e7-a051-6eb8942074ae None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:18.645858 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 522/1051] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:15:18 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 10:15:18.647424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 530/1052] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:15:18.649254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2183da3-66ab-4bb9-a6bb-9d419d931ced None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:18.651978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08a32b-b5d8-41c1-a3f8-9264e4587cc0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:18.652266 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08a32b-b5d8-41c1-a3f8-9264e4587cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:18.652540 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca08a32b-b5d8-41c1-a3f8-9264e4587cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:18.664275 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:18.669681 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ca08a32b-b5d8-41c1-a3f8-9264e4587cc0 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:18.671050 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ca08a32b-b5d8-41c1-a3f8-9264e4587cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:18.747824 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 531/1053] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:18 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 10:15:18.758486 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2183da3-66ab-4bb9-a6bb-9d419d931ced None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:18.767452 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08a32b-b5d8-41c1-a3f8-9264e4587cc0 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:18.795072 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e4a72-a80a-4b5c-9027-b34b032af8c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:18.795923 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e4a72-a80a-4b5c-9027-b34b032af8c1 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:18.796209 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e4a72-a80a-4b5c-9027-b34b032af8c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:18.796570 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16e4a72-a80a-4b5c-9027-b34b032af8c1 None admin] PATH_INFO: `/v3/projects/5895f47d0ba5452d96a7f3114983eb0a` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:18.797761 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e4a72-a80a-4b5c-9027-b34b032af8c1 None admin] RBAC: Authorizing `identity:get_project(project_id=5895f47d0ba5452d96a7f3114983eb0a)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:18.799205 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16e4a72-a80a-4b5c-9027-b34b032af8c1 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:18.800941 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 523/1054] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:15:18 2022] GET /identity/v3/projects/5895f47d0ba5452d96a7f3114983eb0a => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:18.806957 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e4a72-a80a-4b5c-9027-b34b032af8c1 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:18.873812 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca08a32b-b5d8-41c1-a3f8-9264e4587cc0 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:18.893963 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:18.900628 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:18.911002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43993e68-3407-4a33-96d3-89586a374c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:18.911817 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43993e68-3407-4a33-96d3-89586a374c23 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:18.912100 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43993e68-3407-4a33-96d3-89586a374c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:18.912395 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43993e68-3407-4a33-96d3-89586a374c23 None admin] PATH_INFO: `/v3/services/image` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:18.913024 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43993e68-3407-4a33-96d3-89586a374c23 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:18.914300 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16e4a72-a80a-4b5c-9027-b34b032af8c1 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:18.914467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43993e68-3407-4a33-96d3-89586a374c23 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:18.926110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-43993e68-3407-4a33-96d3-89586a374c23 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 10:15:18.926110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:18.926110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 10:15:18.926110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 10:15:18.926110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 10:15:18.926110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 10:15:18.926110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 10:15:18.926110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:15:18.926110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 10:15:18.926110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:18.926110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:15:18.926110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:18.926110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:18.926110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:18.926110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:18.926110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:18.926110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:18.927219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:18.927219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:18.927219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:18.927219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:18.927219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:18.927219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:18.927219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 10:15:18.927219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 10:15:18.927219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 10:15:18.927219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 10:15:18.927219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:18.927219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:18.927219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:18.927219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:18.927219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:18.927219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:18.928140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:18.928140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:18.928140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:18.928140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:18.928140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:18.928140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:18.928140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:18.928140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:18.928140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:18.928140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:18.928140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 10:15:18.928140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:15:18.928140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 30 10:15:18.928140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:18.932242 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 532/1055] 10.222.0.47 () {56 vars in 1105 bytes} [Thu Jun 30 10:15:18 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:15:18.933091 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43993e68-3407-4a33-96d3-89586a374c23 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:18.953745 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0bc2a-bcf1-4139-a7ba-7436d908a113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:18.954337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0bc2a-bcf1-4139-a7ba-7436d908a113 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:18.954605 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0bc2a-bcf1-4139-a7ba-7436d908a113 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:18.954879 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f0bc2a-bcf1-4139-a7ba-7436d908a113 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:18.955754 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0bc2a-bcf1-4139-a7ba-7436d908a113 None admin] RBAC: Adding query filter params (role.id=e63bda042e104a4a93fba15ffcfe1183, scope.project.id=5895f47d0ba5452d96a7f3114983eb0a, user.id=ddbf7d7fd1bd4931bf22b3985fc0bdc6) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:18.956064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0bc2a-bcf1-4139-a7ba-7436d908a113 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:18.957498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f0bc2a-bcf1-4139-a7ba-7436d908a113 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:18.971349 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 524/1056] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:15:18 2022] GET /identity/v3/role_assignments?user.id=ddbf7d7fd1bd4931bf22b3985fc0bdc6&scope.project.id=5895f47d0ba5452d96a7f3114983eb0a&role.id=e63bda042e104a4a93fba15ffcfe1183 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:19.051905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43993e68-3407-4a33-96d3-89586a374c23 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:19.088675 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b75be-0e6e-458b-9886-f6c3965c5ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:19.089364 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b75be-0e6e-458b-9886-f6c3965c5ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:19.089645 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b75be-0e6e-458b-9886-f6c3965c5ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:19.090248 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117b75be-0e6e-458b-9886-f6c3965c5ea0 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:19.091008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117b75be-0e6e-458b-9886-f6c3965c5ea0 None admin] RBAC: Adding query filter params (name=image) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:19.091344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117b75be-0e6e-458b-9886-f6c3965c5ea0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:19.092746 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117b75be-0e6e-458b-9886-f6c3965c5ea0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:19.103346 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 533/1057] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:15:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:19.105444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0bc2a-bcf1-4139-a7ba-7436d908a113 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:19.212051 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f0bc2a-bcf1-4139-a7ba-7436d908a113 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:19.246174 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c9a791-3f38-4244-ac92-5ef1f77a9b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:19.246846 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9a791-3f38-4244-ac92-5ef1f77a9b82 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:19.247130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9a791-3f38-4244-ac92-5ef1f77a9b82 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:19.247438 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9a791-3f38-4244-ac92-5ef1f77a9b82 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:19.248145 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c9a791-3f38-4244-ac92-5ef1f77a9b82 None admin] RBAC: Adding query filter params (type=image) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:19.248452 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c9a791-3f38-4244-ac92-5ef1f77a9b82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:19.249893 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c9a791-3f38-4244-ac92-5ef1f77a9b82 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:19.255043 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 525/1058] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:15:19 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:19.260464 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b75be-0e6e-458b-9886-f6c3965c5ea0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:19.316379 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:19.365440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117b75be-0e6e-458b-9886-f6c3965c5ea0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:19.401234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162dd91-2308-4faf-b9a3-2f20e8f8ba68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:19.401902 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162dd91-2308-4faf-b9a3-2f20e8f8ba68 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:19.402186 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162dd91-2308-4faf-b9a3-2f20e8f8ba68 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:19.402463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162dd91-2308-4faf-b9a3-2f20e8f8ba68 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:19.403213 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e162dd91-2308-4faf-b9a3-2f20e8f8ba68 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:19.404617 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e162dd91-2308-4faf-b9a3-2f20e8f8ba68 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:19.491658 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 534/1059] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jun 30 10:15:19 2022] POST /identity/v3/endpoints => generated 327 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:15:19.642917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417456d-57d1-449c-8bec-0eca5e198ac6 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:19.643127 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417456d-57d1-449c-8bec-0eca5e198ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:19.644707 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7417456d-57d1-449c-8bec-0eca5e198ac6 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:19.645114 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 526/1060] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:15:19.650939 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac7771-66fc-479f-9a1a-fa8ab8778d72 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:19.651201 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac7771-66fc-479f-9a1a-fa8ab8778d72 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:19.651522 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac7771-66fc-479f-9a1a-fa8ab8778d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:19.668885 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-caac7771-66fc-479f-9a1a-fa8ab8778d72 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:19.670097 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-caac7771-66fc-479f-9a1a-fa8ab8778d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:19.745965 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 535/1061] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:15:19.768904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417456d-57d1-449c-8bec-0eca5e198ac6 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:19.780171 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:19.878093 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7417456d-57d1-449c-8bec-0eca5e198ac6 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:19.914502 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054231fb-7e15-487e-8d39-766308923aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:19.915546 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054231fb-7e15-487e-8d39-766308923aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:19.915982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054231fb-7e15-487e-8d39-766308923aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:19.916407 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054231fb-7e15-487e-8d39-766308923aa3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:19.927124 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054231fb-7e15-487e-8d39-766308923aa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:15:19.935569 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-054231fb-7e15-487e-8d39-766308923aa3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 10:15:19.935569 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:19.935569 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:19.935569 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:19.935569 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:19.935569 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:19.935569 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:19.935569 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:19.935569 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:19.935569 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:19.935569 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:19.935569 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:19.935569 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:15:19.935569 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:15:19.935569 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:15:19.935569 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:15:19.936691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:15:19.936691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:15:19.936691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:15:19.936691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:15:19.936691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:19.936691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:19.936691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:19.936691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:19.936691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:19.936691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:19.936691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:19.936691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:19.936691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:19.936691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:19.936691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:19.936691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:19.937795 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:19.937795 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:19.937795 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:19.937795 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:19.937795 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:15:19.937795 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:15:19.937795 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:15:19.937795 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:15:19.937795 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:15:19.937795 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:15:19.937795 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 10:15:19.937795 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:19.940202 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 527/1062] 10.222.0.47 () {56 vars in 1120 bytes} [Thu Jun 30 10:15:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:15:19.945965 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac7771-66fc-479f-9a1a-fa8ab8778d72 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:19.981774 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:20.046752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac7771-66fc-479f-9a1a-fa8ab8778d72 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:20.082405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82197509-d5ff-4637-a0ce-96414dfafa5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:20.083080 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82197509-d5ff-4637-a0ce-96414dfafa5c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:20.083395 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82197509-d5ff-4637-a0ce-96414dfafa5c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:20.083676 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82197509-d5ff-4637-a0ce-96414dfafa5c None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:20.084432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82197509-d5ff-4637-a0ce-96414dfafa5c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:20.084744 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82197509-d5ff-4637-a0ce-96414dfafa5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:20.086127 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82197509-d5ff-4637-a0ce-96414dfafa5c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:20.099471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 536/1063] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Jun 30 10:15:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:20.101076 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054231fb-7e15-487e-8d39-766308923aa3 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:20.172557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:20.206589 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054231fb-7e15-487e-8d39-766308923aa3 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:20.240866 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109f2b9-41fc-4da9-8673-3e5de6b431e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:20.241661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109f2b9-41fc-4da9-8673-3e5de6b431e9 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:20.241994 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109f2b9-41fc-4da9-8673-3e5de6b431e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:20.242275 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a109f2b9-41fc-4da9-8673-3e5de6b431e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:20.253595 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a109f2b9-41fc-4da9-8673-3e5de6b431e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:20.255041 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a109f2b9-41fc-4da9-8673-3e5de6b431e9 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:20.273077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-a109f2b9-41fc-4da9-8673-3e5de6b431e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:15:20.273077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:20.273077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:20.273077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:20.273077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:20.273077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:20.273077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:20.273077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:20.273077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:20.273077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:20.273077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:20.273077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:20.273077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:15:20.273077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:15:20.273077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:15:20.273985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:15:20.273985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:20.273985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:20.273985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:20.273985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:20.273985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:20.273985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:20.273985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:20.273985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:20.273985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:20.273985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:20.273985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:20.273985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:20.273985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:20.273985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:20.273985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:20.273985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:20.274888 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:15:20.274888 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:15:20.274888 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:15:20.274888 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:20.275800 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 528/1064] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:15:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:15:20.280146 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82197509-d5ff-4637-a0ce-96414dfafa5c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:20.377582 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:20.384117 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82197509-d5ff-4637-a0ce-96414dfafa5c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:20.418191 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a506df-cc94-4ad8-883b-7bc8ea431470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:20.418857 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a506df-cc94-4ad8-883b-7bc8ea431470 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:20.419144 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a506df-cc94-4ad8-883b-7bc8ea431470 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:20.419454 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a506df-cc94-4ad8-883b-7bc8ea431470 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:20.420179 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a506df-cc94-4ad8-883b-7bc8ea431470 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:20.420490 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a506df-cc94-4ad8-883b-7bc8ea431470 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:20.421892 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a506df-cc94-4ad8-883b-7bc8ea431470 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:20.438278 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 537/1065] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:20.439573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109f2b9-41fc-4da9-8673-3e5de6b431e9 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:20.549831 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a109f2b9-41fc-4da9-8673-3e5de6b431e9 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:20.570181 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:20.588356 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56762b-8646-429d-a8e3-bf8015ebc38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:20.589168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56762b-8646-429d-a8e3-bf8015ebc38b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:20.589455 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56762b-8646-429d-a8e3-bf8015ebc38b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:20.589734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a56762b-8646-429d-a8e3-bf8015ebc38b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:20.598341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56762b-8646-429d-a8e3-bf8015ebc38b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:20.600095 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a56762b-8646-429d-a8e3-bf8015ebc38b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:20.615629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-4a56762b-8646-429d-a8e3-bf8015ebc38b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 10:15:20.615629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:20.615629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:20.615629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:20.615629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:20.615629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:20.615629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:20.615629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:20.615629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:20.615629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:20.615629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:20.615629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:20.615629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:15:20.615629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:15:20.615629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:15:20.615629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:15:20.616817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:20.616817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:20.616817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:15:20.616817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:15:20.616817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:15:20.616817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:15:20.616817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:20.616817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:20.616817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:20.616817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:20.616817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:20.616817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:20.616817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:20.616817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:20.616817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:20.616817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:20.616817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:20.616817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:20.617977 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:20.617977 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:20.617977 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:15:20.617977 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:15:20.617977 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:15:20.617977 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:15:20.617977 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:15:20.617977 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:15:20.617977 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 10:15:20.617977 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:20.623341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 529/1066] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jun 30 10:15:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:15:20.626446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a506df-cc94-4ad8-883b-7bc8ea431470 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:20.737346 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a506df-cc94-4ad8-883b-7bc8ea431470 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:20.770570 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aedc0b-5587-4ffe-a8b6-214d2ea95a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:20.771245 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aedc0b-5587-4ffe-a8b6-214d2ea95a50 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:20.771561 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aedc0b-5587-4ffe-a8b6-214d2ea95a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:20.772099 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aedc0b-5587-4ffe-a8b6-214d2ea95a50 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:20.772942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aedc0b-5587-4ffe-a8b6-214d2ea95a50 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:20.773256 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aedc0b-5587-4ffe-a8b6-214d2ea95a50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:20.774718 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aedc0b-5587-4ffe-a8b6-214d2ea95a50 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:20.796978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 538/1067] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jun 30 10:15:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:20.798551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56762b-8646-429d-a8e3-bf8015ebc38b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:20.900842 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a56762b-8646-429d-a8e3-bf8015ebc38b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:20.935360 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0fb5a7-61da-46ca-8803-a4074d5f7908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:20.936500 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fb5a7-61da-46ca-8803-a4074d5f7908 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:20.937124 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fb5a7-61da-46ca-8803-a4074d5f7908 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:20.937666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fb5a7-61da-46ca-8803-a4074d5f7908 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:20.948815 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0fb5a7-61da-46ca-8803-a4074d5f7908 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:20.950546 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0fb5a7-61da-46ca-8803-a4074d5f7908 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:20.968061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-2e0fb5a7-61da-46ca-8803-a4074d5f7908 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:15:20.968061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:20.968061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:20.968061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:20.968061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:20.968061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:20.968061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:20.968061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:20.968061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:20.968061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:20.968061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:20.968061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:20.968061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:15:20.968061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:15:20.968061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:15:20.969631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:15:20.969631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:20.969631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:20.969631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:20.969631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:20.969631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:20.969631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:20.969631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:20.969631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:20.969631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:20.969631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:20.969631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:20.969631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:20.969631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:20.969631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:20.969631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:20.969631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:20.970520 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:15:20.970520 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:15:20.970520 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:15:20.970520 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:20.977197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 530/1068] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:15:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:15:20.978259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aedc0b-5587-4ffe-a8b6-214d2ea95a50 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:20.995463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:21.081753 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aedc0b-5587-4ffe-a8b6-214d2ea95a50 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:21.094513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4144c-9f7c-4376-90b1-52f36df538c3 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:21.095195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4144c-9f7c-4376-90b1-52f36df538c3 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:21.095529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4144c-9f7c-4376-90b1-52f36df538c3 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:21.097093 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 531/1069] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:15:21.102609 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d712a8-5895-4083-9bb9-634e74dc3aeb None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:21.102895 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d712a8-5895-4083-9bb9-634e74dc3aeb None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:21.103171 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d712a8-5895-4083-9bb9-634e74dc3aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:21.120159 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-16d712a8-5895-4083-9bb9-634e74dc3aeb None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:21.120720 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba95f0b-c3a7-40fc-94d9-2a9a5962eec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:21.121338 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-16d712a8-5895-4083-9bb9-634e74dc3aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:21.121441 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba95f0b-c3a7-40fc-94d9-2a9a5962eec8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:21.121670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba95f0b-c3a7-40fc-94d9-2a9a5962eec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:21.121952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba95f0b-c3a7-40fc-94d9-2a9a5962eec8 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:21.122682 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba95f0b-c3a7-40fc-94d9-2a9a5962eec8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:21.122997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba95f0b-c3a7-40fc-94d9-2a9a5962eec8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:21.124608 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba95f0b-c3a7-40fc-94d9-2a9a5962eec8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:21.136881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 539/1070] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:21.142980 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba95f0b-c3a7-40fc-94d9-2a9a5962eec8 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:21.152307 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:21.197281 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 532/1071] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 10:15:21.221806 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d712a8-5895-4083-9bb9-634e74dc3aeb None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:21.226331 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:21.252118 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba95f0b-c3a7-40fc-94d9-2a9a5962eec8 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:21.291246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcdccc6-7fe2-48dc-bd1b-802cf43a5499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:21.292182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdccc6-7fe2-48dc-bd1b-802cf43a5499 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:21.292468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdccc6-7fe2-48dc-bd1b-802cf43a5499 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:21.292860 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcdccc6-7fe2-48dc-bd1b-802cf43a5499 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:21.300063 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcdccc6-7fe2-48dc-bd1b-802cf43a5499 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:21.301625 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcdccc6-7fe2-48dc-bd1b-802cf43a5499 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:21.317178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-7bcdccc6-7fe2-48dc-bd1b-802cf43a5499 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:15:21.317178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:21.317178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:21.317178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:21.317178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:21.317178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:21.317178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:21.317178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:21.317178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:21.317178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:21.317178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:21.317178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:21.317178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:15:21.317178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:15:21.317178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:15:21.318088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:15:21.318088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:21.318088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:21.318088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:21.318088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:21.318088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:21.318088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:21.318088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:21.318088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:21.318088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:21.318088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:21.318088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:21.318088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:21.318088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:21.318088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:21.318088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:21.318088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:21.318998 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:15:21.318998 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:15:21.318998 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:15:21.318998 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:15:21.318998 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:15:21.318998 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:15:21.318998 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:15:21.318998 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:21.318998 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 540/1072] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:15:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:15:21.323960 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d712a8-5895-4083-9bb9-634e74dc3aeb None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:21.324405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcdccc6-7fe2-48dc-bd1b-802cf43a5499 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:21.359913 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac5f0a-c6e7-4f28-955e-10ce261e6aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:21.360717 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac5f0a-c6e7-4f28-955e-10ce261e6aea None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:21.361004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac5f0a-c6e7-4f28-955e-10ce261e6aea None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:21.361281 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ac5f0a-c6e7-4f28-955e-10ce261e6aea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:21.371610 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ac5f0a-c6e7-4f28-955e-10ce261e6aea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:15:21.379325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-11ac5f0a-c6e7-4f28-955e-10ce261e6aea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:15:21.379325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:21.379325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:21.379325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:21.379325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:21.379325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:21.379325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:21.379325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:21.379325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:21.379325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:21.379325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:21.379325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:21.379325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:15:21.379325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:15:21.379325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:15:21.379325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:15:21.380280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:15:21.380280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:15:21.380280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:15:21.380280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:15:21.380280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:21.380280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:21.380280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:21.380280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:21.380280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:21.380280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:21.380280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:21.380280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:21.380280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:21.380280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:21.380280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:21.380280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:21.381146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:21.381146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:21.381146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:21.381146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:21.381146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:15:21.381146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:15:21.381146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:15:21.381146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:15:21.381146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:15:21.381146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:15:21.381146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:15:21.381146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:21.381146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 533/1073] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:15:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:15:21.386645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac5f0a-c6e7-4f28-955e-10ce261e6aea None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:21.417204 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:21.438978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcdccc6-7fe2-48dc-bd1b-802cf43a5499 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:21.475042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cacb8b-89cf-4e18-a713-eb5fd2abe65a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:21.475852 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cacb8b-89cf-4e18-a713-eb5fd2abe65a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:21.476139 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cacb8b-89cf-4e18-a713-eb5fd2abe65a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:21.476439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cacb8b-89cf-4e18-a713-eb5fd2abe65a None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:21.477234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cacb8b-89cf-4e18-a713-eb5fd2abe65a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:21.477547 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cacb8b-89cf-4e18-a713-eb5fd2abe65a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:21.478983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cacb8b-89cf-4e18-a713-eb5fd2abe65a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:21.480627 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:21.491347 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 541/1074] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:15:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:21.497132 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cacb8b-89cf-4e18-a713-eb5fd2abe65a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:21.517292 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ac5f0a-c6e7-4f28-955e-10ce261e6aea None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:21.555341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8266c65-d25d-4f66-b52b-d674812d0ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:21.556125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8266c65-d25d-4f66-b52b-d674812d0ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:21.556408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8266c65-d25d-4f66-b52b-d674812d0ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:21.556817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8266c65-d25d-4f66-b52b-d674812d0ef3 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:21.557548 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8266c65-d25d-4f66-b52b-d674812d0ef3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:21.557859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8266c65-d25d-4f66-b52b-d674812d0ef3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:21.559369 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8266c65-d25d-4f66-b52b-d674812d0ef3 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:21.573032 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 534/1075] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:15:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:21.574066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8266c65-d25d-4f66-b52b-d674812d0ef3 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:21.599121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cacb8b-89cf-4e18-a713-eb5fd2abe65a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:21.632484 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105c1d43-d350-4f15-852e-78a8adeb9e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:21.633375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c1d43-d350-4f15-852e-78a8adeb9e6f None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:21.633659 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c1d43-d350-4f15-852e-78a8adeb9e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:21.633943 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105c1d43-d350-4f15-852e-78a8adeb9e6f None admin] PATH_INFO: `/v3/projects/50902e0539434c82b505dc5009aeaf85/users/71f6d7715e6f49ffb9fdf6e1307bbd51/roles/0b2dccc298c3428f91cea20447956c6a` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:21.636231 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105c1d43-d350-4f15-852e-78a8adeb9e6f None admin] RBAC: Authorizing `identity:create_grant(project_id=50902e0539434c82b505dc5009aeaf85, user_id=71f6d7715e6f49ffb9fdf6e1307bbd51, role_id=0b2dccc298c3428f91cea20447956c6a)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:21.637678 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105c1d43-d350-4f15-852e-78a8adeb9e6f None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:21.660820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:21.675603 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8266c65-d25d-4f66-b52b-d674812d0ef3 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:21.703238 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 542/1076] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jun 30 10:15:21 2022] PUT /identity/v3/projects/50902e0539434c82b505dc5009aeaf85/users/71f6d7715e6f49ffb9fdf6e1307bbd51/roles/0b2dccc298c3428f91cea20447956c6a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:15:21.704581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b026ce-6f4a-4f96-b6b1-f890a7f87b45 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:21.704869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b026ce-6f4a-4f96-b6b1-f890a7f87b45 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:21.705146 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b026ce-6f4a-4f96-b6b1-f890a7f87b45 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:21.706444 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 543/1077] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:15:21.710876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd4b7f-5e35-4cc8-a3a9-de72aa507019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:21.712128 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd4b7f-5e35-4cc8-a3a9-de72aa507019 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:21.712417 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd4b7f-5e35-4cc8-a3a9-de72aa507019 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:21.712698 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd4b7f-5e35-4cc8-a3a9-de72aa507019 None admin] PATH_INFO: `/v3/users/ddbf7d7fd1bd4931bf22b3985fc0bdc6` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:21.714057 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd4b7f-5e35-4cc8-a3a9-de72aa507019 None admin] RBAC: Authorizing `identity:get_user(user_id=ddbf7d7fd1bd4931bf22b3985fc0bdc6)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:21.714853 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969afcb2-8da1-4ec3-abe3-44d95968762f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:21.715208 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969afcb2-8da1-4ec3-abe3-44d95968762f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:21.715582 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969afcb2-8da1-4ec3-abe3-44d95968762f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:21.715988 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd4b7f-5e35-4cc8-a3a9-de72aa507019 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:21.722984 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 535/1078] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:15:21 2022] GET /identity/v3/users/ddbf7d7fd1bd4931bf22b3985fc0bdc6 => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:21.724004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd4b7f-5e35-4cc8-a3a9-de72aa507019 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:21.733911 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-969afcb2-8da1-4ec3-abe3-44d95968762f None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:21.735554 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-969afcb2-8da1-4ec3-abe3-44d95968762f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:21.790753 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:21.809807 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 544/1079] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 10:15:21.833866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969afcb2-8da1-4ec3-abe3-44d95968762f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:21.843772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd4b7f-5e35-4cc8-a3a9-de72aa507019 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:21.892001 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39c9a5-a567-464b-989f-7ce4b9f011c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:21.893409 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39c9a5-a567-464b-989f-7ce4b9f011c6 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:21.893992 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39c9a5-a567-464b-989f-7ce4b9f011c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:21.894276 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39c9a5-a567-464b-989f-7ce4b9f011c6 None admin] PATH_INFO: `/v3/projects/5895f47d0ba5452d96a7f3114983eb0a` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:21.895813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d39c9a5-a567-464b-989f-7ce4b9f011c6 None admin] RBAC: Authorizing `identity:get_project(project_id=5895f47d0ba5452d96a7f3114983eb0a)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:21.898205 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d39c9a5-a567-464b-989f-7ce4b9f011c6 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:21.903313 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 536/1080] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:15:21 2022] GET /identity/v3/projects/5895f47d0ba5452d96a7f3114983eb0a => generated 311 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:21.906328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39c9a5-a567-464b-989f-7ce4b9f011c6 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:21.931414 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:21.950460 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969afcb2-8da1-4ec3-abe3-44d95968762f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:21.988432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76386e-69a4-49b3-955e-7d349bd78137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:21.989088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76386e-69a4-49b3-955e-7d349bd78137 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:21.989374 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76386e-69a4-49b3-955e-7d349bd78137 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:21.989653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76386e-69a4-49b3-955e-7d349bd78137 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:22.021536 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39c9a5-a567-464b-989f-7ce4b9f011c6 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:22.048560 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:22.059222 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19bfa09-e914-439e-bd49-18a905bc055e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:22.060150 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bfa09-e914-439e-bd49-18a905bc055e None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:22.060435 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bfa09-e914-439e-bd49-18a905bc055e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:22.060711 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19bfa09-e914-439e-bd49-18a905bc055e None admin] PATH_INFO: `/v3/projects/5895f47d0ba5452d96a7f3114983eb0a/users/ddbf7d7fd1bd4931bf22b3985fc0bdc6/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:22.063492 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19bfa09-e914-439e-bd49-18a905bc055e None admin] RBAC: Authorizing `identity:create_grant(project_id=5895f47d0ba5452d96a7f3114983eb0a, user_id=ddbf7d7fd1bd4931bf22b3985fc0bdc6, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:22.064938 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19bfa09-e914-439e-bd49-18a905bc055e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:22.065879 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76386e-69a4-49b3-955e-7d349bd78137 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:22.067532 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76386e-69a4-49b3-955e-7d349bd78137 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:22.136430 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 537/1081] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jun 30 10:15:21 2022] PUT /identity/v3/projects/5895f47d0ba5452d96a7f3114983eb0a/users/ddbf7d7fd1bd4931bf22b3985fc0bdc6/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:15:22.174217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 545/1082] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:15:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 10:15:22.180335 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19bfa09-e914-439e-bd49-18a905bc055e None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:22.274088 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:22.297994 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19bfa09-e914-439e-bd49-18a905bc055e None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:22.333496 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058059c9-9b70-4fcc-b227-fbe50d702dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:22.334302 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058059c9-9b70-4fcc-b227-fbe50d702dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:22.334509 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058059c9-9b70-4fcc-b227-fbe50d702dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:22.334788 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058059c9-9b70-4fcc-b227-fbe50d702dd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:22.346443 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058059c9-9b70-4fcc-b227-fbe50d702dd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:22.347931 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058059c9-9b70-4fcc-b227-fbe50d702dd5 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:22.365419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-058059c9-9b70-4fcc-b227-fbe50d702dd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:15:22.365419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:22.365419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:22.365419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:22.365419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:22.365419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:22.365419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:22.365419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:22.365419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:22.365419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:22.365419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:22.365419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:22.365419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:15:22.365419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:15:22.365419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:15:22.366351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:15:22.366351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:22.366351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:22.366351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:22.366351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:22.366351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:22.366351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:22.366351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:22.366351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:22.366351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:22.366351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:22.366351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:22.366351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:22.366351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:22.366351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:22.366351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:22.366351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:22.367255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:15:22.367255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:15:22.367255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:15:22.367255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:22.367255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 538/1083] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:15:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:15:22.374003 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76386e-69a4-49b3-955e-7d349bd78137 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:22.458375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:22.474769 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76386e-69a4-49b3-955e-7d349bd78137 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:22.516435 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce934ffb-120c-4323-8695-86fa4202b318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:22.517103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce934ffb-120c-4323-8695-86fa4202b318 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:22.517386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce934ffb-120c-4323-8695-86fa4202b318 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:22.517661 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce934ffb-120c-4323-8695-86fa4202b318 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:22.518382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce934ffb-120c-4323-8695-86fa4202b318 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:22.518689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce934ffb-120c-4323-8695-86fa4202b318 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:22.520090 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce934ffb-120c-4323-8695-86fa4202b318 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:22.532539 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 546/1084] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:23.779733 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd60a5-6568-4df3-8496-7b52c6817521 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:23.780249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd60a5-6568-4df3-8496-7b52c6817521 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:23.780249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd60a5-6568-4df3-8496-7b52c6817521 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:23.782284 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 539/1085] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:15:23.787498 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4be98-9d8d-428b-9f84-a2e96e08902d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:23.787913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4be98-9d8d-428b-9f84-a2e96e08902d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:23.788180 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c4be98-9d8d-428b-9f84-a2e96e08902d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:23.804614 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-17c4be98-9d8d-428b-9f84-a2e96e08902d None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:23.805791 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-17c4be98-9d8d-428b-9f84-a2e96e08902d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:23.871220 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 547/1086] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:15:23.897206 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd60a5-6568-4df3-8496-7b52c6817521 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:23.998488 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd60a5-6568-4df3-8496-7b52c6817521 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:24.027726 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:24.032420 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8da477-bb88-48c3-b701-41f4b282e2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:24.033173 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8da477-bb88-48c3-b701-41f4b282e2db None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:24.033458 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8da477-bb88-48c3-b701-41f4b282e2db None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:24.033734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8da477-bb88-48c3-b701-41f4b282e2db None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:24.044001 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8da477-bb88-48c3-b701-41f4b282e2db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:15:24.052219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-4a8da477-bb88-48c3-b701-41f4b282e2db None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 10:15:24.052219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:24.052219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:24.052219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:24.052219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:24.052219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:24.052219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:24.052219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:24.052219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:24.052219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:24.052219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:24.052219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:24.052219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:15:24.052219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:15:24.052219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:15:24.052219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:15:24.053328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:15:24.053328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:15:24.053328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:15:24.053328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:15:24.053328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:24.053328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:24.053328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:24.053328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:24.053328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:24.053328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:24.053328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:24.053328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:24.053328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:24.053328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:24.053328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:24.053328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:24.054198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:24.054198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:24.054198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:24.054198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:24.054198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:15:24.054198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:15:24.054198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:15:24.054198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:15:24.054198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:15:24.054198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:15:24.054198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 30 10:15:24.054198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:24.054198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 540/1087] 10.222.0.47 () {56 vars in 1120 bytes} [Thu Jun 30 10:15:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:15:24.059770 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c4be98-9d8d-428b-9f84-a2e96e08902d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:24.134873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:24.159398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c4be98-9d8d-428b-9f84-a2e96e08902d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:24.198244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a17a9-4053-4adf-a380-901d1d9e4d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:24.198919 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a17a9-4053-4adf-a380-901d1d9e4d13 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:24.199202 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a17a9-4053-4adf-a380-901d1d9e4d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:24.200243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a17a9-4053-4adf-a380-901d1d9e4d13 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:24.201000 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a17a9-4053-4adf-a380-901d1d9e4d13 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:24.201306 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a17a9-4053-4adf-a380-901d1d9e4d13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:24.203376 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a17a9-4053-4adf-a380-901d1d9e4d13 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:24.204822 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b926f6-870b-43b9-be5b-53e8ed0e28a5 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:24.205137 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b926f6-870b-43b9-be5b-53e8ed0e28a5 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:24.205421 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b926f6-870b-43b9-be5b-53e8ed0e28a5 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:24.206786 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 541/1088] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:15:24.211750 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056113fc-37ec-4bcf-81e2-1f5811a697b7 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:24.211884 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 548/1089] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Jun 30 10:15:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:24.212038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056113fc-37ec-4bcf-81e2-1f5811a697b7 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:24.212313 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056113fc-37ec-4bcf-81e2-1f5811a697b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:24.217184 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a17a9-4053-4adf-a380-901d1d9e4d13 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:24.228730 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-056113fc-37ec-4bcf-81e2-1f5811a697b7 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:24.229922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-056113fc-37ec-4bcf-81e2-1f5811a697b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:24.298752 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 542/1090] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 30 10:15:24.318099 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a17a9-4053-4adf-a380-901d1d9e4d13 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:24.322064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056113fc-37ec-4bcf-81e2-1f5811a697b7 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:24.352016 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e79c4-b2ee-4edb-a686-7cc60ade099a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:24.352805 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e79c4-b2ee-4edb-a686-7cc60ade099a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:24.353093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e79c4-b2ee-4edb-a686-7cc60ade099a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:24.353372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e79c4-b2ee-4edb-a686-7cc60ade099a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:24.356730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:24.364654 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e79c4-b2ee-4edb-a686-7cc60ade099a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:24.366106 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e79c4-b2ee-4edb-a686-7cc60ade099a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:24.383334 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-569e79c4-b2ee-4edb-a686-7cc60ade099a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:15:24.383334 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:24.383334 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:24.383334 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:24.383334 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:24.383334 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:24.383334 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:24.383334 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:24.383334 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:24.383334 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:24.383334 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:24.383334 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:24.383334 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:15:24.383334 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:15:24.383334 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:15:24.384385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:15:24.384385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:24.384385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:24.384385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:24.384385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:24.384385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:24.384385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:24.384385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:24.384385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:24.384385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:24.384385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:24.384385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:24.384385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:24.384385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:24.384385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:24.384385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:24.384385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:24.385309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:15:24.385309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:15:24.385309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:15:24.385309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:24.385309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 549/1091] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:15:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:15:24.390851 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e79c4-b2ee-4edb-a686-7cc60ade099a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:24.422854 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056113fc-37ec-4bcf-81e2-1f5811a697b7 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:24.445234 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:24.456865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a5c209-3c8b-4599-88a4-8008e5ee02f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:24.457656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5c209-3c8b-4599-88a4-8008e5ee02f4 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:24.457945 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5c209-3c8b-4599-88a4-8008e5ee02f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:24.458298 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a5c209-3c8b-4599-88a4-8008e5ee02f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:24.468850 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a5c209-3c8b-4599-88a4-8008e5ee02f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:15:24.476922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-d6a5c209-3c8b-4599-88a4-8008e5ee02f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:15:24.476922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:24.476922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:24.476922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:24.476922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:24.476922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:24.476922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:24.476922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:24.476922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:24.476922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:24.476922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:24.476922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:24.476922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:15:24.476922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:15:24.476922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:15:24.476922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:15:24.478653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:15:24.478653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:15:24.478653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:15:24.478653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:15:24.478653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:24.478653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:24.478653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:24.478653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:24.478653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:24.478653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:24.478653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:24.478653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:24.478653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:24.478653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:24.478653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:24.478653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:24.480317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:24.480317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:24.480317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:24.480317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:24.480317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:15:24.480317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:15:24.480317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:15:24.480317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:15:24.480317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:15:24.480317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:15:24.480317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:15:24.480317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:24.480317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 543/1092] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:15:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:15:24.483999 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a5c209-3c8b-4599-88a4-8008e5ee02f4 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:24.493710 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e79c4-b2ee-4edb-a686-7cc60ade099a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:24.527644 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df351024-54d4-4fc6-9c5c-dcdb67f9e4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:24.528312 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df351024-54d4-4fc6-9c5c-dcdb67f9e4ee None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:24.528599 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df351024-54d4-4fc6-9c5c-dcdb67f9e4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:24.528877 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df351024-54d4-4fc6-9c5c-dcdb67f9e4ee None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:24.529601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df351024-54d4-4fc6-9c5c-dcdb67f9e4ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:24.529943 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df351024-54d4-4fc6-9c5c-dcdb67f9e4ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:24.531998 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df351024-54d4-4fc6-9c5c-dcdb67f9e4ee None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:24.543597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 550/1093] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:24.549866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df351024-54d4-4fc6-9c5c-dcdb67f9e4ee None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:24.594591 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a5c209-3c8b-4599-88a4-8008e5ee02f4 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:24.599067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:24.631283 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786712b0-26d0-4dae-b5ca-b6f0d4d97646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:24.631976 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786712b0-26d0-4dae-b5ca-b6f0d4d97646 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:24.632259 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786712b0-26d0-4dae-b5ca-b6f0d4d97646 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:24.632607 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786712b0-26d0-4dae-b5ca-b6f0d4d97646 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:24.633350 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786712b0-26d0-4dae-b5ca-b6f0d4d97646 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:24.633645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786712b0-26d0-4dae-b5ca-b6f0d4d97646 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:24.635000 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786712b0-26d0-4dae-b5ca-b6f0d4d97646 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:24.643728 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 544/1094] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:15:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:24.649404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786712b0-26d0-4dae-b5ca-b6f0d4d97646 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:24.650414 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df351024-54d4-4fc6-9c5c-dcdb67f9e4ee None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:24.684846 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2d908-2dfd-452b-8def-0094ba43de1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:24.685632 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2d908-2dfd-452b-8def-0094ba43de1a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:24.685917 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2d908-2dfd-452b-8def-0094ba43de1a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:24.686195 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2d908-2dfd-452b-8def-0094ba43de1a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:24.695458 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:24.696349 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d2d908-2dfd-452b-8def-0094ba43de1a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:24.697820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d2d908-2dfd-452b-8def-0094ba43de1a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:24.712267 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-c3d2d908-2dfd-452b-8def-0094ba43de1a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 30 10:15:24.712267 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:24.712267 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:24.712267 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:24.712267 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:24.712267 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:24.712267 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:24.712267 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:24.712267 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:24.712267 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:24.712267 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:24.712267 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:24.712267 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:15:24.712267 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:15:24.712267 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:15:24.712267 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:15:24.713976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:24.713976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:24.713976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:15:24.713976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:15:24.713976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:15:24.713976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:15:24.713976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:24.713976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:24.713976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:24.713976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:24.713976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:24.713976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:24.713976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:24.713976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:24.713976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:24.713976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:24.713976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:24.713976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:24.714968 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:24.714968 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:24.714968 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:15:24.714968 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:15:24.714968 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:15:24.714968 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:15:24.714968 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:15:24.714968 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:15:24.714968 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 30 10:15:24.714968 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:24.714968 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 551/1095] 10.222.0.47 () {56 vars in 1093 bytes} [Thu Jun 30 10:15:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:15:24.720053 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2d908-2dfd-452b-8def-0094ba43de1a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:24.751252 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786712b0-26d0-4dae-b5ca-b6f0d4d97646 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:24.785065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ba1d9-f49e-4814-82c2-c8abda5471a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:24.785934 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ba1d9-f49e-4814-82c2-c8abda5471a8 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:24.786217 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ba1d9-f49e-4814-82c2-c8abda5471a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:24.786493 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53ba1d9-f49e-4814-82c2-c8abda5471a8 None admin] PATH_INFO: `/v3/users/ddbf7d7fd1bd4931bf22b3985fc0bdc6` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:24.787730 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53ba1d9-f49e-4814-82c2-c8abda5471a8 None admin] RBAC: Authorizing `identity:get_user(user_id=ddbf7d7fd1bd4931bf22b3985fc0bdc6)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:24.789169 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53ba1d9-f49e-4814-82c2-c8abda5471a8 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:24.790793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 545/1096] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:15:24 2022] GET /identity/v3/users/ddbf7d7fd1bd4931bf22b3985fc0bdc6 => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:24.797122 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ba1d9-f49e-4814-82c2-c8abda5471a8 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:24.821443 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2d908-2dfd-452b-8def-0094ba43de1a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:24.830922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:24.855832 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d3602-b7c2-4ac6-8d6f-94956628730a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:24.856501 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d3602-b7c2-4ac6-8d6f-94956628730a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:24.856972 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d3602-b7c2-4ac6-8d6f-94956628730a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:24.857546 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d3602-b7c2-4ac6-8d6f-94956628730a None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:24.859681 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d3602-b7c2-4ac6-8d6f-94956628730a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:24.860083 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d3602-b7c2-4ac6-8d6f-94956628730a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:24.861554 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d3602-b7c2-4ac6-8d6f-94956628730a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:24.878182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 552/1097] 10.222.0.47 () {56 vars in 1133 bytes} [Thu Jun 30 10:15:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:24.883821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d3602-b7c2-4ac6-8d6f-94956628730a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:24.895531 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53ba1d9-f49e-4814-82c2-c8abda5471a8 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:24.909628 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:24.930966 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e738e1e-b9cb-4340-9653-3c417beb7b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:24.931803 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e738e1e-b9cb-4340-9653-3c417beb7b83 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:24.932140 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e738e1e-b9cb-4340-9653-3c417beb7b83 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:24.932822 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e738e1e-b9cb-4340-9653-3c417beb7b83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:24.982257 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d3602-b7c2-4ac6-8d6f-94956628730a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:24.999136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e738e1e-b9cb-4340-9653-3c417beb7b83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:25.000662 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e738e1e-b9cb-4340-9653-3c417beb7b83 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:25.015422 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31436100-c5b1-40ef-981a-bc085c2ba103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:25.016329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31436100-c5b1-40ef-981a-bc085c2ba103 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:25.016621 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31436100-c5b1-40ef-981a-bc085c2ba103 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:25.016896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31436100-c5b1-40ef-981a-bc085c2ba103 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:25.049694 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:25.083349 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31436100-c5b1-40ef-981a-bc085c2ba103 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:25.084759 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31436100-c5b1-40ef-981a-bc085c2ba103 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:25.098111 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 546/1098] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:15:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 10:15:25.103865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e738e1e-b9cb-4340-9653-3c417beb7b83 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:25.116704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:25.182004 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 553/1099] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:15:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 10:15:25.187645 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31436100-c5b1-40ef-981a-bc085c2ba103 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:25.207116 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e738e1e-b9cb-4340-9653-3c417beb7b83 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:25.241337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f65bc2-cb2e-4b06-8482-ef5dc1cec207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:25.242189 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f65bc2-cb2e-4b06-8482-ef5dc1cec207 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:25.242472 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f65bc2-cb2e-4b06-8482-ef5dc1cec207 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:25.242753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f65bc2-cb2e-4b06-8482-ef5dc1cec207 None admin] PATH_INFO: `/v3/projects/5895f47d0ba5452d96a7f3114983eb0a` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:25.243942 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f65bc2-cb2e-4b06-8482-ef5dc1cec207 None admin] RBAC: Authorizing `identity:get_project(project_id=5895f47d0ba5452d96a7f3114983eb0a)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:25.245365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f65bc2-cb2e-4b06-8482-ef5dc1cec207 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:25.247095 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 547/1100] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:15:25 2022] GET /identity/v3/projects/5895f47d0ba5452d96a7f3114983eb0a => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:25.251871 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:25.253323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f65bc2-cb2e-4b06-8482-ef5dc1cec207 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:25.286759 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31436100-c5b1-40ef-981a-bc085c2ba103 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:25.319635 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c9d57-f94e-4c61-a615-d9462be7d579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:25.320428 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c9d57-f94e-4c61-a615-d9462be7d579 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:25.320721 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c9d57-f94e-4c61-a615-d9462be7d579 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:25.321001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08c9d57-f94e-4c61-a615-d9462be7d579 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:25.331932 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c9d57-f94e-4c61-a615-d9462be7d579 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:25.332400 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:25.333439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08c9d57-f94e-4c61-a615-d9462be7d579 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:25.350749 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-b08c9d57-f94e-4c61-a615-d9462be7d579 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:15:25.350749 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:25.350749 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:25.350749 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:25.350749 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:25.350749 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:25.350749 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:25.350749 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:25.350749 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:25.350749 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:25.350749 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:25.350749 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:25.350749 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:15:25.350749 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:15:25.350749 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:15:25.351859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:15:25.351859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:25.351859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:25.351859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:25.351859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:25.351859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:25.351859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:25.351859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:25.351859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:25.351859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:25.351859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:25.351859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:25.351859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:25.351859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:25.351859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:25.351859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:25.351859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:25.352911 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:15:25.352911 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:15:25.352911 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:15:25.352911 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:25.353150 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f65bc2-cb2e-4b06-8482-ef5dc1cec207 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:25.353249 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 554/1101] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:15:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:15:25.357659 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c9d57-f94e-4c61-a615-d9462be7d579 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:25.386903 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb621e-fed6-45a0-9464-da26eece849e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:25.387590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb621e-fed6-45a0-9464-da26eece849e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:25.387876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb621e-fed6-45a0-9464-da26eece849e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:25.388158 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efb621e-fed6-45a0-9464-da26eece849e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:25.388991 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb621e-fed6-45a0-9464-da26eece849e None admin] RBAC: Adding query filter params (role.id=e63bda042e104a4a93fba15ffcfe1183, scope.project.id=5895f47d0ba5452d96a7f3114983eb0a, user.id=ddbf7d7fd1bd4931bf22b3985fc0bdc6) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:25.389306 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb621e-fed6-45a0-9464-da26eece849e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:25.390967 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efb621e-fed6-45a0-9464-da26eece849e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:25.397106 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 548/1102] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:15:25 2022] GET /identity/v3/role_assignments?user.id=ddbf7d7fd1bd4931bf22b3985fc0bdc6&scope.project.id=5895f47d0ba5452d96a7f3114983eb0a&role.id=e63bda042e104a4a93fba15ffcfe1183 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:25.464000 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08c9d57-f94e-4c61-a615-d9462be7d579 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:25.482562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:25.501775 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313660d-c65a-4a5c-90d3-1ca367d9ea07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:25.502560 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313660d-c65a-4a5c-90d3-1ca367d9ea07 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:25.502919 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313660d-c65a-4a5c-90d3-1ca367d9ea07 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:25.503302 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313660d-c65a-4a5c-90d3-1ca367d9ea07 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:25.504114 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0313660d-c65a-4a5c-90d3-1ca367d9ea07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:25.504485 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0313660d-c65a-4a5c-90d3-1ca367d9ea07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:25.505899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0313660d-c65a-4a5c-90d3-1ca367d9ea07 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:25.520032 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 555/1103] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:15:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:25.525992 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb621e-fed6-45a0-9464-da26eece849e None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:25.632998 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efb621e-fed6-45a0-9464-da26eece849e None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:25.668258 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47d258-6023-471d-9794-ca852a13d39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:25.669054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47d258-6023-471d-9794-ca852a13d39d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:25.669340 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47d258-6023-471d-9794-ca852a13d39d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:25.669616 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47d258-6023-471d-9794-ca852a13d39d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:25.677206 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f47d258-6023-471d-9794-ca852a13d39d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:25.678641 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f47d258-6023-471d-9794-ca852a13d39d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:25.693136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-7f47d258-6023-471d-9794-ca852a13d39d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:15:25.693136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:25.693136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:25.693136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:25.693136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:25.693136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:25.693136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:25.693136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:25.693136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:25.693136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:25.693136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:25.693136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:25.693136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:15:25.693136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:15:25.693136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:15:25.694130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:15:25.694130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:25.694130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:25.694130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:25.694130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:25.694130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:25.694130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:25.694130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:25.694130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:25.694130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:25.694130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:25.694130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:25.694130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:25.694130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:25.694130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:25.694130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:25.694130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:25.695073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:15:25.695073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:15:25.695073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:15:25.695073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:15:25.695073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:15:25.695073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:15:25.695073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:15:25.695073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:25.695073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 549/1104] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:15:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:15:25.699752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313660d-c65a-4a5c-90d3-1ca367d9ea07 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:25.800166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313660d-c65a-4a5c-90d3-1ca367d9ea07 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:25.834293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e73c1-3f12-4a55-9abd-5f745889a8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:25.834944 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e73c1-3f12-4a55-9abd-5f745889a8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:25.835225 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e73c1-3f12-4a55-9abd-5f745889a8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:25.835527 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e73c1-3f12-4a55-9abd-5f745889a8d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:25.836314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e73c1-3f12-4a55-9abd-5f745889a8d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:25.836619 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e73c1-3f12-4a55-9abd-5f745889a8d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:25.838039 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e73c1-3f12-4a55-9abd-5f745889a8d9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:25.850408 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 556/1105] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:15:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:25.856401 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47d258-6023-471d-9794-ca852a13d39d None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:25.861755 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:25.960311 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47d258-6023-471d-9794-ca852a13d39d None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:25.992953 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb8de1-036f-46f5-8593-79ce164e9b8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:25.993601 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb8de1-036f-46f5-8593-79ce164e9b8e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:25.993884 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb8de1-036f-46f5-8593-79ce164e9b8e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:25.994159 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb8de1-036f-46f5-8593-79ce164e9b8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:25.994980 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bb8de1-036f-46f5-8593-79ce164e9b8e None admin] RBAC: Adding query filter params (role.id=0b2dccc298c3428f91cea20447956c6a, scope.project.id=50902e0539434c82b505dc5009aeaf85, user.id=71f6d7715e6f49ffb9fdf6e1307bbd51) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:25.995315 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bb8de1-036f-46f5-8593-79ce164e9b8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:25.996734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bb8de1-036f-46f5-8593-79ce164e9b8e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:26.003075 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 550/1106] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:15:25 2022] GET /identity/v3/role_assignments?user.id=71f6d7715e6f49ffb9fdf6e1307bbd51&scope.project.id=50902e0539434c82b505dc5009aeaf85&role.id=0b2dccc298c3428f91cea20447956c6a => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:27.506377 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b6f40-d0c0-4ba0-8bf3-34b7ef0683c1 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:27.506887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b6f40-d0c0-4ba0-8bf3-34b7ef0683c1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:27.506995 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b6f40-d0c0-4ba0-8bf3-34b7ef0683c1 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:27.508352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 557/1107] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:15:27.513475 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e04cf-dc65-4306-aa89-bbbab20754e7 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:27.513911 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e04cf-dc65-4306-aa89-bbbab20754e7 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:27.514034 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e04cf-dc65-4306-aa89-bbbab20754e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:27.531906 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-b75e04cf-dc65-4306-aa89-bbbab20754e7 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:27.533074 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-b75e04cf-dc65-4306-aa89-bbbab20754e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:27.592689 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:27.598614 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 551/1108] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:15:27.623856 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b6f40-d0c0-4ba0-8bf3-34b7ef0683c1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:27.666009 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:27.727804 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b6f40-d0c0-4ba0-8bf3-34b7ef0683c1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:27.760908 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c28579-a468-43ca-bf82-c492f6ad40b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:27.761625 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c28579-a468-43ca-bf82-c492f6ad40b6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:27.761905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c28579-a468-43ca-bf82-c492f6ad40b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:27.762176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c28579-a468-43ca-bf82-c492f6ad40b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:27.763366 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c28579-a468-43ca-bf82-c492f6ad40b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:27.764868 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c28579-a468-43ca-bf82-c492f6ad40b6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:27.766588 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 558/1109] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:15:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:27.772601 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e04cf-dc65-4306-aa89-bbbab20754e7 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:27.868697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e04cf-dc65-4306-aa89-bbbab20754e7 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:27.901109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f411370-e469-48ad-9e7d-ffa09025ddbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:27.901768 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f411370-e469-48ad-9e7d-ffa09025ddbd None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:27.902051 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f411370-e469-48ad-9e7d-ffa09025ddbd None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:27.902325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f411370-e469-48ad-9e7d-ffa09025ddbd None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:27.903037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f411370-e469-48ad-9e7d-ffa09025ddbd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:27.904503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f411370-e469-48ad-9e7d-ffa09025ddbd None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:27.976898 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 552/1110] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jun 30 10:15:27 2022] POST /identity/v3/users => generated 286 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:15:30.080670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa07ea6-4d4e-4212-9072-e695ee8219ee None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:30.081223 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa07ea6-4d4e-4212-9072-e695ee8219ee None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:30.081223 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa07ea6-4d4e-4212-9072-e695ee8219ee None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:30.082590 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 559/1111] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:15:30.087762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337686e8-8627-4bc7-a530-82ae602af91d None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:30.088157 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337686e8-8627-4bc7-a530-82ae602af91d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:30.088313 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337686e8-8627-4bc7-a530-82ae602af91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:30.105392 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-337686e8-8627-4bc7-a530-82ae602af91d None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:30.106559 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-337686e8-8627-4bc7-a530-82ae602af91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:30.155044 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:30.172833 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 553/1112] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:15:30.196715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa07ea6-4d4e-4212-9072-e695ee8219ee None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:30.294938 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa07ea6-4d4e-4212-9072-e695ee8219ee None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:30.302502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:30.328288 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e604155-e25a-4b91-a67a-4501a49bf2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:30.329127 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e604155-e25a-4b91-a67a-4501a49bf2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:30.329407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e604155-e25a-4b91-a67a-4501a49bf2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:30.329680 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e604155-e25a-4b91-a67a-4501a49bf2c9 None admin] PATH_INFO: `/v3/roles/284d1573e06d47eba0196afbe92d4d7d` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:30.331173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e604155-e25a-4b91-a67a-4501a49bf2c9 None admin] RBAC: Authorizing `identity:get_role(role_id=284d1573e06d47eba0196afbe92d4d7d)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:30.332515 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e604155-e25a-4b91-a67a-4501a49bf2c9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:30.336646 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 560/1113] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:15:30 2022] GET /identity/v3/roles/284d1573e06d47eba0196afbe92d4d7d => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:30.339599 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337686e8-8627-4bc7-a530-82ae602af91d None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:30.434965 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337686e8-8627-4bc7-a530-82ae602af91d None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:30.467548 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48717adf-7461-49a0-9d64-ca0d6bd85814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:30.468948 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48717adf-7461-49a0-9d64-ca0d6bd85814 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:30.469224 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48717adf-7461-49a0-9d64-ca0d6bd85814 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:30.469496 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48717adf-7461-49a0-9d64-ca0d6bd85814 None admin] PATH_INFO: `/v3/users/7f8cd3b71ccb41048792cbb7c9017a24` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:30.482043 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48717adf-7461-49a0-9d64-ca0d6bd85814 None admin] RBAC: Authorizing `identity:get_user(user_id=7f8cd3b71ccb41048792cbb7c9017a24)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:30.483579 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48717adf-7461-49a0-9d64-ca0d6bd85814 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:30.485390 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 554/1114] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:15:30 2022] GET /identity/v3/users/7f8cd3b71ccb41048792cbb7c9017a24 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:30.491056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e604155-e25a-4b91-a67a-4501a49bf2c9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:30.589024 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e604155-e25a-4b91-a67a-4501a49bf2c9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:30.622073 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c999bf-3a18-429c-83ad-ecd4229f0fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:30.622769 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c999bf-3a18-429c-83ad-ecd4229f0fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:30.623059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c999bf-3a18-429c-83ad-ecd4229f0fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:30.623366 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c999bf-3a18-429c-83ad-ecd4229f0fe0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:30.664370 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:30.688363 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c999bf-3a18-429c-83ad-ecd4229f0fe0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:30.689808 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c999bf-3a18-429c-83ad-ecd4229f0fe0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:30.785253 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 561/1115] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:15:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 10:15:30.791065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48717adf-7461-49a0-9d64-ca0d6bd85814 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:30.833319 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:30.888483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48717adf-7461-49a0-9d64-ca0d6bd85814 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:30.922052 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05bc31-2200-4144-a111-37e88da3f056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:30.922894 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05bc31-2200-4144-a111-37e88da3f056 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:30.923176 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05bc31-2200-4144-a111-37e88da3f056 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:30.923480 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05bc31-2200-4144-a111-37e88da3f056 None admin] PATH_INFO: `/v3/projects/5895f47d0ba5452d96a7f3114983eb0a` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:30.924650 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c05bc31-2200-4144-a111-37e88da3f056 None admin] RBAC: Authorizing `identity:get_project(project_id=5895f47d0ba5452d96a7f3114983eb0a)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:30.926074 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c05bc31-2200-4144-a111-37e88da3f056 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:30.927771 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 555/1116] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:15:30 2022] GET /identity/v3/projects/5895f47d0ba5452d96a7f3114983eb0a => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:30.933784 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c999bf-3a18-429c-83ad-ecd4229f0fe0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:31.005681 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:31.029821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c999bf-3a18-429c-83ad-ecd4229f0fe0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:31.062952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c94d7c-1d27-4a6a-a81a-8763ebc956d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:31.063615 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c94d7c-1d27-4a6a-a81a-8763ebc956d1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:31.063898 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c94d7c-1d27-4a6a-a81a-8763ebc956d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:31.064198 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c94d7c-1d27-4a6a-a81a-8763ebc956d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:31.065071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c94d7c-1d27-4a6a-a81a-8763ebc956d1 None admin] RBAC: Adding query filter params (role.id=284d1573e06d47eba0196afbe92d4d7d, scope.project.id=5895f47d0ba5452d96a7f3114983eb0a, user.id=7f8cd3b71ccb41048792cbb7c9017a24) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:31.065438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c94d7c-1d27-4a6a-a81a-8763ebc956d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:31.066906 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c94d7c-1d27-4a6a-a81a-8763ebc956d1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:31.072170 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 562/1117] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:15:30 2022] GET /identity/v3/role_assignments?user.id=7f8cd3b71ccb41048792cbb7c9017a24&scope.project.id=5895f47d0ba5452d96a7f3114983eb0a&role.id=284d1573e06d47eba0196afbe92d4d7d => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:33.090623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d27e0-0d5c-43f4-9d11-873e8f7f6c34 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:33.091161 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d27e0-0d5c-43f4-9d11-873e8f7f6c34 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:33.091161 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d27e0-0d5c-43f4-9d11-873e8f7f6c34 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:33.093081 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 556/1118] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:15:33.098580 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3535fd-de7c-404f-9870-58dca227598c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:33.098986 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3535fd-de7c-404f-9870-58dca227598c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:33.099144 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3535fd-de7c-404f-9870-58dca227598c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:33.116428 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4a3535fd-de7c-404f-9870-58dca227598c None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:33.117614 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4a3535fd-de7c-404f-9870-58dca227598c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:33.181764 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 563/1119] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:15:33.205620 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d27e0-0d5c-43f4-9d11-873e8f7f6c34 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:33.304454 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d27e0-0d5c-43f4-9d11-873e8f7f6c34 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:33.315192 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:33.337186 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cc296-a781-4747-81df-1f3d74eed8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:33.338031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cc296-a781-4747-81df-1f3d74eed8e2 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:33.338316 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cc296-a781-4747-81df-1f3d74eed8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:33.338594 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cc296-a781-4747-81df-1f3d74eed8e2 None admin] PATH_INFO: `/v3/roles/284d1573e06d47eba0196afbe92d4d7d` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:33.340099 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3cc296-a781-4747-81df-1f3d74eed8e2 None admin] RBAC: Authorizing `identity:get_role(role_id=284d1573e06d47eba0196afbe92d4d7d)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:33.341510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3cc296-a781-4747-81df-1f3d74eed8e2 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:33.343002 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 557/1120] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:15:33 2022] GET /identity/v3/roles/284d1573e06d47eba0196afbe92d4d7d => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:33.348677 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3535fd-de7c-404f-9870-58dca227598c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:33.416944 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:33.450465 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3535fd-de7c-404f-9870-58dca227598c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:33.482291 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0476375-a577-4d0e-a5fd-3766d160fc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:33.483157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0476375-a577-4d0e-a5fd-3766d160fc10 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:33.483995 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0476375-a577-4d0e-a5fd-3766d160fc10 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:33.484286 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0476375-a577-4d0e-a5fd-3766d160fc10 None admin] PATH_INFO: `/v3/users/7f8cd3b71ccb41048792cbb7c9017a24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:33.485489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0476375-a577-4d0e-a5fd-3766d160fc10 None admin] RBAC: Authorizing `identity:get_user(user_id=7f8cd3b71ccb41048792cbb7c9017a24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:33.486924 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0476375-a577-4d0e-a5fd-3766d160fc10 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:33.488602 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 564/1121] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:15:33 2022] GET /identity/v3/users/7f8cd3b71ccb41048792cbb7c9017a24 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:33.494331 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cc296-a781-4747-81df-1f3d74eed8e2 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:33.590794 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cc296-a781-4747-81df-1f3d74eed8e2 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:33.623980 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9998dd-4d8c-4734-9f3a-3b25e0ae38e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:33.624770 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9998dd-4d8c-4734-9f3a-3b25e0ae38e2 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:33.625052 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9998dd-4d8c-4734-9f3a-3b25e0ae38e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:33.625325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9998dd-4d8c-4734-9f3a-3b25e0ae38e2 None admin] PATH_INFO: `/v3/projects/5895f47d0ba5452d96a7f3114983eb0a` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:33.626496 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9998dd-4d8c-4734-9f3a-3b25e0ae38e2 None admin] RBAC: Authorizing `identity:get_project(project_id=5895f47d0ba5452d96a7f3114983eb0a)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:33.627947 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9998dd-4d8c-4734-9f3a-3b25e0ae38e2 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:33.629610 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 558/1122] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:15:33 2022] GET /identity/v3/projects/5895f47d0ba5452d96a7f3114983eb0a => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:33.635985 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0476375-a577-4d0e-a5fd-3766d160fc10 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:33.731849 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0476375-a577-4d0e-a5fd-3766d160fc10 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:33.766092 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2192cf-774e-4695-a344-087c436000dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:33.766957 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2192cf-774e-4695-a344-087c436000dc None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:33.767240 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2192cf-774e-4695-a344-087c436000dc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:33.767548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2192cf-774e-4695-a344-087c436000dc None admin] PATH_INFO: `/v3/projects/5895f47d0ba5452d96a7f3114983eb0a/users/7f8cd3b71ccb41048792cbb7c9017a24/roles/284d1573e06d47eba0196afbe92d4d7d` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:33.769734 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2192cf-774e-4695-a344-087c436000dc None admin] RBAC: Authorizing `identity:create_grant(project_id=5895f47d0ba5452d96a7f3114983eb0a, user_id=7f8cd3b71ccb41048792cbb7c9017a24, role_id=284d1573e06d47eba0196afbe92d4d7d)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:33.771160 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2192cf-774e-4695-a344-087c436000dc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:33.774390 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:33.883326 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 565/1123] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jun 30 10:15:33 2022] PUT /identity/v3/projects/5895f47d0ba5452d96a7f3114983eb0a/users/7f8cd3b71ccb41048792cbb7c9017a24/roles/284d1573e06d47eba0196afbe92d4d7d => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:15:35.899482 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db01db9-7cd3-43e5-84f4-8842855fd603 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:35.900000 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db01db9-7cd3-43e5-84f4-8842855fd603 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:35.900000 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db01db9-7cd3-43e5-84f4-8842855fd603 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:35.901344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 559/1124] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:15:35.907627 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08180cb0-b780-4e3a-90ff-631ef8e40282 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:35.908033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08180cb0-b780-4e3a-90ff-631ef8e40282 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:35.908184 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08180cb0-b780-4e3a-90ff-631ef8e40282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:35.925330 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-08180cb0-b780-4e3a-90ff-631ef8e40282 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:35.926501 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-08180cb0-b780-4e3a-90ff-631ef8e40282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:35.992107 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 566/1125] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:15:36.018053 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db01db9-7cd3-43e5-84f4-8842855fd603 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:36.067844 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:36.119530 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db01db9-7cd3-43e5-84f4-8842855fd603 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:36.153001 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1cd854-7669-4782-8ff7-18b6c7946b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:36.153796 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cd854-7669-4782-8ff7-18b6c7946b1e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:36.154082 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cd854-7669-4782-8ff7-18b6c7946b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:36.154368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cd854-7669-4782-8ff7-18b6c7946b1e None admin] PATH_INFO: `/v3/roles/284d1573e06d47eba0196afbe92d4d7d` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:36.156075 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1cd854-7669-4782-8ff7-18b6c7946b1e None admin] RBAC: Authorizing `identity:get_role(role_id=284d1573e06d47eba0196afbe92d4d7d)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:36.157437 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1cd854-7669-4782-8ff7-18b6c7946b1e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:36.158793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 560/1126] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:15:36 2022] GET /identity/v3/roles/284d1573e06d47eba0196afbe92d4d7d => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:36.164596 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08180cb0-b780-4e3a-90ff-631ef8e40282 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:36.259586 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08180cb0-b780-4e3a-90ff-631ef8e40282 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:36.284002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:36.292090 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4cd367-b7bf-4557-8a3d-c516db18f5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:36.292944 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4cd367-b7bf-4557-8a3d-c516db18f5dd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:36.297491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4cd367-b7bf-4557-8a3d-c516db18f5dd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:36.297491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4cd367-b7bf-4557-8a3d-c516db18f5dd None admin] PATH_INFO: `/v3/users/7f8cd3b71ccb41048792cbb7c9017a24` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:36.297491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4cd367-b7bf-4557-8a3d-c516db18f5dd None admin] RBAC: Authorizing `identity:get_user(user_id=7f8cd3b71ccb41048792cbb7c9017a24)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:36.297491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4cd367-b7bf-4557-8a3d-c516db18f5dd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:36.298449 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 567/1127] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:15:36 2022] GET /identity/v3/users/7f8cd3b71ccb41048792cbb7c9017a24 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:36.304082 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1cd854-7669-4782-8ff7-18b6c7946b1e None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:36.401873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1cd854-7669-4782-8ff7-18b6c7946b1e None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:36.415577 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:36.435028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95143cf4-e76e-42e8-ac4c-57ea78ca957a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:36.435702 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95143cf4-e76e-42e8-ac4c-57ea78ca957a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:36.435993 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95143cf4-e76e-42e8-ac4c-57ea78ca957a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:36.436325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95143cf4-e76e-42e8-ac4c-57ea78ca957a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:36.502472 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95143cf4-e76e-42e8-ac4c-57ea78ca957a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:36.503968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95143cf4-e76e-42e8-ac4c-57ea78ca957a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:36.598559 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 561/1128] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:15:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 30 10:15:36.604409 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4cd367-b7bf-4557-8a3d-c516db18f5dd None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:36.699781 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4cd367-b7bf-4557-8a3d-c516db18f5dd None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:36.731384 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbcc454-c5e7-4925-abf5-4ae6bef10b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:36.732231 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcc454-c5e7-4925-abf5-4ae6bef10b35 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:36.732512 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcc454-c5e7-4925-abf5-4ae6bef10b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:36.732783 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbcc454-c5e7-4925-abf5-4ae6bef10b35 None admin] PATH_INFO: `/v3/projects/5895f47d0ba5452d96a7f3114983eb0a` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:36.734044 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbcc454-c5e7-4925-abf5-4ae6bef10b35 None admin] RBAC: Authorizing `identity:get_project(project_id=5895f47d0ba5452d96a7f3114983eb0a)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:36.735498 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbcc454-c5e7-4925-abf5-4ae6bef10b35 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:36.737156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 568/1129] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:15:36 2022] GET /identity/v3/projects/5895f47d0ba5452d96a7f3114983eb0a => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:36.743059 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95143cf4-e76e-42e8-ac4c-57ea78ca957a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:36.758019 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:36.840173 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95143cf4-e76e-42e8-ac4c-57ea78ca957a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:36.872386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d19de8-bbc6-4929-b742-2a71624beddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:36.873031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d19de8-bbc6-4929-b742-2a71624beddb None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:36.873310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d19de8-bbc6-4929-b742-2a71624beddb None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:36.873582 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d19de8-bbc6-4929-b742-2a71624beddb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:36.875060 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d19de8-bbc6-4929-b742-2a71624beddb None admin] RBAC: Adding query filter params (role.id=284d1573e06d47eba0196afbe92d4d7d, scope.project.id=5895f47d0ba5452d96a7f3114983eb0a, user.id=7f8cd3b71ccb41048792cbb7c9017a24) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:36.875411 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d19de8-bbc6-4929-b742-2a71624beddb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:36.876822 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d19de8-bbc6-4929-b742-2a71624beddb None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:36.882390 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 562/1130] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:15:36 2022] GET /identity/v3/role_assignments?user.id=7f8cd3b71ccb41048792cbb7c9017a24&scope.project.id=5895f47d0ba5452d96a7f3114983eb0a&role.id=284d1573e06d47eba0196afbe92d4d7d => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:38.991288 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12e9b5-1b56-4751-a8a5-b16f596c3c38 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:38.991836 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12e9b5-1b56-4751-a8a5-b16f596c3c38 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:38.991836 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b12e9b5-1b56-4751-a8a5-b16f596c3c38 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:38.993219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 569/1131] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:15:38.999928 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06f80c-1e0d-4d33-a731-d1ada255b29d None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:39.000399 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06f80c-1e0d-4d33-a731-d1ada255b29d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:39.000489 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06f80c-1e0d-4d33-a731-d1ada255b29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:39.017685 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-1e06f80c-1e0d-4d33-a731-d1ada255b29d None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:39.018876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-1e06f80c-1e0d-4d33-a731-d1ada255b29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:39.093544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 563/1132] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:15:39.118119 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12e9b5-1b56-4751-a8a5-b16f596c3c38 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:39.197038 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:39.235601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b12e9b5-1b56-4751-a8a5-b16f596c3c38 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:39.267893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08dd5423-762c-47b8-9f37-8ae001eb4a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:39.268609 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd5423-762c-47b8-9f37-8ae001eb4a9d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:39.268889 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd5423-762c-47b8-9f37-8ae001eb4a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:39.269159 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08dd5423-762c-47b8-9f37-8ae001eb4a9d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:39.270308 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd5423-762c-47b8-9f37-8ae001eb4a9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:39.271772 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08dd5423-762c-47b8-9f37-8ae001eb4a9d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:39.273787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 570/1133] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:15:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:39.279615 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06f80c-1e0d-4d33-a731-d1ada255b29d None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:39.290603 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:39.378039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06f80c-1e0d-4d33-a731-d1ada255b29d None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:39.411741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618a104c-1efa-44d7-8ee8-045c3f068c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:39.412400 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a104c-1efa-44d7-8ee8-045c3f068c4d None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:39.412688 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a104c-1efa-44d7-8ee8-045c3f068c4d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:39.412971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618a104c-1efa-44d7-8ee8-045c3f068c4d None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:39.413686 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618a104c-1efa-44d7-8ee8-045c3f068c4d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:39.415125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618a104c-1efa-44d7-8ee8-045c3f068c4d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:39.497703 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 564/1134] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jun 30 10:15:39 2022] POST /identity/v3/projects => generated 311 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:15:41.548716 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f479e-b0fe-492c-b079-db5be99e9643 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:41.549194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f479e-b0fe-492c-b079-db5be99e9643 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:41.549281 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398f479e-b0fe-492c-b079-db5be99e9643 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:41.550636 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 571/1135] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:15:41.555727 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250118a-d00e-4f57-8e92-cfdc59d394e4 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:41.556143 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250118a-d00e-4f57-8e92-cfdc59d394e4 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:41.556298 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4250118a-d00e-4f57-8e92-cfdc59d394e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:41.573259 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-4250118a-d00e-4f57-8e92-cfdc59d394e4 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:41.574449 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-4250118a-d00e-4f57-8e92-cfdc59d394e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:41.628612 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:41.639353 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 565/1136] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:15:41.663522 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f479e-b0fe-492c-b079-db5be99e9643 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:41.761611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398f479e-b0fe-492c-b079-db5be99e9643 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:41.788603 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:41.794142 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556e9c9-ee0c-4974-a364-b8384256888d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:41.794963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556e9c9-ee0c-4974-a364-b8384256888d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:41.795247 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556e9c9-ee0c-4974-a364-b8384256888d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:41.796156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556e9c9-ee0c-4974-a364-b8384256888d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:41.799004 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556e9c9-ee0c-4974-a364-b8384256888d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:41.799004 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556e9c9-ee0c-4974-a364-b8384256888d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:41.800509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 572/1137] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:15:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:41.806253 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250118a-d00e-4f57-8e92-cfdc59d394e4 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:41.901657 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4250118a-d00e-4f57-8e92-cfdc59d394e4 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:41.934716 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0dc82c-4cc8-49e6-9fae-915e92de5a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:41.935405 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0dc82c-4cc8-49e6-9fae-915e92de5a19 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:41.935687 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0dc82c-4cc8-49e6-9fae-915e92de5a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:41.936001 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0dc82c-4cc8-49e6-9fae-915e92de5a19 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:41.936756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0dc82c-4cc8-49e6-9fae-915e92de5a19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:41.938175 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0dc82c-4cc8-49e6-9fae-915e92de5a19 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:42.015334 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 566/1138] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jun 30 10:15:41 2022] POST /identity/v3/users => generated 286 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:15:44.155097 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62c2c7-e621-4ec4-bc25-6e27f48f0d0c None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:44.155706 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62c2c7-e621-4ec4-bc25-6e27f48f0d0c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:44.155821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62c2c7-e621-4ec4-bc25-6e27f48f0d0c None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:44.157180 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 573/1139] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:15:44.162507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865cc23f-56a8-439a-b032-85cd1b7db5e9 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:44.162885 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865cc23f-56a8-439a-b032-85cd1b7db5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:44.163065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865cc23f-56a8-439a-b032-85cd1b7db5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:44.180415 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-865cc23f-56a8-439a-b032-85cd1b7db5e9 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:44.181591 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-865cc23f-56a8-439a-b032-85cd1b7db5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:44.223156 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:44.247183 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 567/1140] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:15:44.271034 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62c2c7-e621-4ec4-bc25-6e27f48f0d0c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:44.379438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62c2c7-e621-4ec4-bc25-6e27f48f0d0c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:44.413879 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c803189-824c-4270-b1c0-79415ea8a13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:44.414590 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c803189-824c-4270-b1c0-79415ea8a13f None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:44.414871 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c803189-824c-4270-b1c0-79415ea8a13f None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:44.415144 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c803189-824c-4270-b1c0-79415ea8a13f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:44.425559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c803189-824c-4270-b1c0-79415ea8a13f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:15:44.433022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-4c803189-824c-4270-b1c0-79415ea8a13f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:15:44.433022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:44.433022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:44.433022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:44.433022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:44.433022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:44.433022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:44.433022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:44.433022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:44.433022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:44.433022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:44.433022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:44.433022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:15:44.433022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:15:44.433022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:15:44.433022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:15:44.434167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:15:44.434167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:15:44.434167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:15:44.434167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:15:44.434167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:44.434167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:44.434167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:44.434167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:44.434167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:44.434167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:44.434167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:44.434167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:44.434167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:44.434167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:44.434167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:44.434167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:44.435315 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:44.435315 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:44.435315 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:44.435315 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:44.435315 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:15:44.435315 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:15:44.435315 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:15:44.435315 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:15:44.435315 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:15:44.435315 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:15:44.435315 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:15:44.435315 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:44.435315 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 574/1141] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:15:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:15:44.440546 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865cc23f-56a8-439a-b032-85cd1b7db5e9 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:44.542798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865cc23f-56a8-439a-b032-85cd1b7db5e9 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:44.575499 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d83cf4-2a19-4943-8621-482cb0eebac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:44.576155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d83cf4-2a19-4943-8621-482cb0eebac7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:44.577065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d83cf4-2a19-4943-8621-482cb0eebac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:44.577626 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d83cf4-2a19-4943-8621-482cb0eebac7 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:44.578364 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d83cf4-2a19-4943-8621-482cb0eebac7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:44.578670 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d83cf4-2a19-4943-8621-482cb0eebac7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:44.580054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d83cf4-2a19-4943-8621-482cb0eebac7 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:44.587752 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 568/1142] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:15:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:44.594019 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c803189-824c-4270-b1c0-79415ea8a13f None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:44.650645 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:44.690854 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c803189-824c-4270-b1c0-79415ea8a13f None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:44.724381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9aa751-a06f-48d7-8a40-26860bc73056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:44.725235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9aa751-a06f-48d7-8a40-26860bc73056 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:44.725515 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9aa751-a06f-48d7-8a40-26860bc73056 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:44.725786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9aa751-a06f-48d7-8a40-26860bc73056 None admin] PATH_INFO: `/v3/users/9d738ac19de4442fa2ec7a1cbc63f270` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:44.738456 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9aa751-a06f-48d7-8a40-26860bc73056 None admin] RBAC: Authorizing `identity:get_user(user_id=9d738ac19de4442fa2ec7a1cbc63f270)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:44.739972 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9aa751-a06f-48d7-8a40-26860bc73056 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:44.741595 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 575/1143] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:15:44 2022] GET /identity/v3/users/9d738ac19de4442fa2ec7a1cbc63f270 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:44.747290 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d83cf4-2a19-4943-8621-482cb0eebac7 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:44.787492 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:44.845271 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d83cf4-2a19-4943-8621-482cb0eebac7 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:44.878481 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e35d10-ca19-4c51-b9fe-e4e46e09cd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:44.879129 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e35d10-ca19-4c51-b9fe-e4e46e09cd65 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:44.879446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e35d10-ca19-4c51-b9fe-e4e46e09cd65 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:44.879726 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e35d10-ca19-4c51-b9fe-e4e46e09cd65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:44.944927 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e35d10-ca19-4c51-b9fe-e4e46e09cd65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:44.946300 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e35d10-ca19-4c51-b9fe-e4e46e09cd65 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:44.982795 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:45.042859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 569/1144] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:15:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 10:15:45.049042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9aa751-a06f-48d7-8a40-26860bc73056 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:45.145701 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9aa751-a06f-48d7-8a40-26860bc73056 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:45.162216 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:45.179695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f7246b-b656-4c16-a676-bcd049cbb5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:45.180554 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7246b-b656-4c16-a676-bcd049cbb5eb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:45.180838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7246b-b656-4c16-a676-bcd049cbb5eb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:45.183584 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f7246b-b656-4c16-a676-bcd049cbb5eb None admin] PATH_INFO: `/v3/projects/27d92635bfcb446d85e3eab74791598d` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:45.183584 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f7246b-b656-4c16-a676-bcd049cbb5eb None admin] RBAC: Authorizing `identity:get_project(project_id=27d92635bfcb446d85e3eab74791598d)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:45.183748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f7246b-b656-4c16-a676-bcd049cbb5eb None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:45.185401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 576/1145] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:15:45 2022] GET /identity/v3/projects/27d92635bfcb446d85e3eab74791598d => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:45.192300 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e35d10-ca19-4c51-b9fe-e4e46e09cd65 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:45.289293 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e35d10-ca19-4c51-b9fe-e4e46e09cd65 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:45.322741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43e1171-1ea4-451a-b707-9559b6251193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:45.323421 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e1171-1ea4-451a-b707-9559b6251193 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:45.323713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e1171-1ea4-451a-b707-9559b6251193 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:45.323994 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43e1171-1ea4-451a-b707-9559b6251193 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:45.324820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e1171-1ea4-451a-b707-9559b6251193 None admin] RBAC: Adding query filter params (role.id=e63bda042e104a4a93fba15ffcfe1183, scope.project.id=27d92635bfcb446d85e3eab74791598d, user.id=9d738ac19de4442fa2ec7a1cbc63f270) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:45.325137 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e1171-1ea4-451a-b707-9559b6251193 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:45.326626 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43e1171-1ea4-451a-b707-9559b6251193 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:45.331968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 570/1146] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:15:45 2022] GET /identity/v3/role_assignments?user.id=9d738ac19de4442fa2ec7a1cbc63f270&scope.project.id=27d92635bfcb446d85e3eab74791598d&role.id=e63bda042e104a4a93fba15ffcfe1183 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:47.353113 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cbb5a-aa23-400f-a374-6cf003a67de8 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:47.353643 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cbb5a-aa23-400f-a374-6cf003a67de8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:47.353643 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6cbb5a-aa23-400f-a374-6cf003a67de8 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:47.355041 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 577/1147] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:15:47.360560 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7fd93-09e9-4866-ab87-30436e40b343 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:47.361022 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7fd93-09e9-4866-ab87-30436e40b343 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:47.361114 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f7fd93-09e9-4866-ab87-30436e40b343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:47.366301 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:47.378581 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-96f7fd93-09e9-4866-ab87-30436e40b343 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:47.383822 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-96f7fd93-09e9-4866-ab87-30436e40b343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:47.447440 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 571/1148] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:15:47.470952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6cbb5a-aa23-400f-a374-6cf003a67de8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:47.570241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6cbb5a-aa23-400f-a374-6cf003a67de8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:47.602834 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1cb538-8195-44de-ab83-a5d35d25c64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:47.603688 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cb538-8195-44de-ab83-a5d35d25c64c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:47.603921 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cb538-8195-44de-ab83-a5d35d25c64c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:47.604195 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1cb538-8195-44de-ab83-a5d35d25c64c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:47.614539 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1cb538-8195-44de-ab83-a5d35d25c64c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:15:47.622003 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-2f1cb538-8195-44de-ab83-a5d35d25c64c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:15:47.622003 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:47.622003 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:47.622003 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:47.622003 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:47.622003 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:47.622003 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:47.622003 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:47.622003 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:47.622003 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:47.622003 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:47.622003 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:47.622003 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:15:47.622003 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:15:47.622003 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:15:47.622003 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:15:47.623199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:15:47.623199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:15:47.623199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:15:47.623199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:15:47.623199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:47.623199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:47.623199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:47.623199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:47.623199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:47.623199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:47.623199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:47.623199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:47.623199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:47.623199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:47.623199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:47.623199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:47.624942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:47.624942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:47.624942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:47.624942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:47.624942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:15:47.624942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:15:47.624942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:15:47.624942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:15:47.624942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:15:47.624942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:15:47.624942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:15:47.624942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:47.624942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 578/1149] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:15:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:15:47.630108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7fd93-09e9-4866-ab87-30436e40b343 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:47.726223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f7fd93-09e9-4866-ab87-30436e40b343 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:47.755998 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:47.760849 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646b075-ff96-491a-8d13-d2a67169eec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:47.761506 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646b075-ff96-491a-8d13-d2a67169eec6 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:47.761792 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646b075-ff96-491a-8d13-d2a67169eec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:47.762067 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9646b075-ff96-491a-8d13-d2a67169eec6 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:47.762809 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646b075-ff96-491a-8d13-d2a67169eec6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:47.763114 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646b075-ff96-491a-8d13-d2a67169eec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:47.765206 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9646b075-ff96-491a-8d13-d2a67169eec6 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:47.773270 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 572/1150] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:15:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:47.779076 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1cb538-8195-44de-ab83-a5d35d25c64c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:47.823453 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:47.876712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1cb538-8195-44de-ab83-a5d35d25c64c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:47.909547 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472d909-d5e7-4d15-a408-8184b581cb42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:47.910410 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472d909-d5e7-4d15-a408-8184b581cb42 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:47.910692 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472d909-d5e7-4d15-a408-8184b581cb42 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:47.910968 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e472d909-d5e7-4d15-a408-8184b581cb42 None admin] PATH_INFO: `/v3/users/9d738ac19de4442fa2ec7a1cbc63f270` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:47.912184 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472d909-d5e7-4d15-a408-8184b581cb42 None admin] RBAC: Authorizing `identity:get_user(user_id=9d738ac19de4442fa2ec7a1cbc63f270)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:47.913613 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e472d909-d5e7-4d15-a408-8184b581cb42 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:47.915224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 579/1151] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:15:47 2022] GET /identity/v3/users/9d738ac19de4442fa2ec7a1cbc63f270 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:47.921281 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646b075-ff96-491a-8d13-d2a67169eec6 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:48.019014 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9646b075-ff96-491a-8d13-d2a67169eec6 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:48.052413 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9004e82b-8b1a-4fcf-8da3-46eefa2f5d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:48.053276 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004e82b-8b1a-4fcf-8da3-46eefa2f5d22 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:48.053559 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004e82b-8b1a-4fcf-8da3-46eefa2f5d22 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:48.053836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9004e82b-8b1a-4fcf-8da3-46eefa2f5d22 None admin] PATH_INFO: `/v3/projects/27d92635bfcb446d85e3eab74791598d` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:48.055010 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9004e82b-8b1a-4fcf-8da3-46eefa2f5d22 None admin] RBAC: Authorizing `identity:get_project(project_id=27d92635bfcb446d85e3eab74791598d)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:48.056475 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9004e82b-8b1a-4fcf-8da3-46eefa2f5d22 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:48.058165 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 573/1152] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:15:47 2022] GET /identity/v3/projects/27d92635bfcb446d85e3eab74791598d => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:15:48.064637 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472d909-d5e7-4d15-a408-8184b581cb42 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:48.160208 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e472d909-d5e7-4d15-a408-8184b581cb42 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:48.172398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:48.192977 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdfd696-b248-4e54-98b7-30f8f4e65c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:48.195316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfd696-b248-4e54-98b7-30f8f4e65c95 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:48.195316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfd696-b248-4e54-98b7-30f8f4e65c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:48.195316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdfd696-b248-4e54-98b7-30f8f4e65c95 None admin] PATH_INFO: `/v3/projects/27d92635bfcb446d85e3eab74791598d/users/9d738ac19de4442fa2ec7a1cbc63f270/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:48.196431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdfd696-b248-4e54-98b7-30f8f4e65c95 None admin] RBAC: Authorizing `identity:create_grant(project_id=27d92635bfcb446d85e3eab74791598d, user_id=9d738ac19de4442fa2ec7a1cbc63f270, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:48.197861 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdfd696-b248-4e54-98b7-30f8f4e65c95 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:48.305367 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 580/1153] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jun 30 10:15:48 2022] PUT /identity/v3/projects/27d92635bfcb446d85e3eab74791598d/users/9d738ac19de4442fa2ec7a1cbc63f270/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:15:50.314004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ba790-1365-40ae-9634-c8bcbd6bbf4b None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:50.314591 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ba790-1365-40ae-9634-c8bcbd6bbf4b None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:50.314591 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319ba790-1365-40ae-9634-c8bcbd6bbf4b None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:50.316152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 574/1154] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:15:50.321354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc79c46-3eac-47c5-bf8d-dcf0c55a71e2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:50.321757 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc79c46-3eac-47c5-bf8d-dcf0c55a71e2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:50.321917 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc79c46-3eac-47c5-bf8d-dcf0c55a71e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:50.338837 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9fc79c46-3eac-47c5-bf8d-dcf0c55a71e2 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:50.340063 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9fc79c46-3eac-47c5-bf8d-dcf0c55a71e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:50.403924 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 581/1155] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:15:50.427119 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ba790-1365-40ae-9634-c8bcbd6bbf4b None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:50.488125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:50.527419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319ba790-1365-40ae-9634-c8bcbd6bbf4b None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:50.559544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae2e97-7f75-4dcd-918d-4fee1ead1b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:50.560326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae2e97-7f75-4dcd-918d-4fee1ead1b6b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:50.561166 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae2e97-7f75-4dcd-918d-4fee1ead1b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:50.561446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae2e97-7f75-4dcd-918d-4fee1ead1b6b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:50.571196 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ae2e97-7f75-4dcd-918d-4fee1ead1b6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:15:50.578741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-a3ae2e97-7f75-4dcd-918d-4fee1ead1b6b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:15:50.578741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:50.578741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:50.578741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:50.578741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:50.578741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:50.578741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:50.578741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:50.578741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:50.578741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:50.578741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:50.578741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:50.578741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:15:50.578741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:15:50.578741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:15:50.578741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:15:50.579876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:15:50.579876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:15:50.579876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:15:50.579876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:15:50.579876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:50.579876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:50.579876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:50.579876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:50.579876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:50.579876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:50.579876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:50.579876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:50.579876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:50.579876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:50.579876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:50.579876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:50.580924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:50.580924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:50.580924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:50.580924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:50.580924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:15:50.580924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:15:50.580924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:15:50.580924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:15:50.580924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:15:50.580924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:15:50.580924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:15:50.580924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:15:50.580924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 575/1156] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:15:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:15:50.586057 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc79c46-3eac-47c5-bf8d-dcf0c55a71e2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:50.681703 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc79c46-3eac-47c5-bf8d-dcf0c55a71e2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:50.683171 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:50.715318 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13940dd3-eaa7-49a6-a0a3-201f6036816e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:50.715978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13940dd3-eaa7-49a6-a0a3-201f6036816e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:50.716263 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13940dd3-eaa7-49a6-a0a3-201f6036816e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:50.716540 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13940dd3-eaa7-49a6-a0a3-201f6036816e None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:50.718045 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13940dd3-eaa7-49a6-a0a3-201f6036816e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:50.718356 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13940dd3-eaa7-49a6-a0a3-201f6036816e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:50.719752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13940dd3-eaa7-49a6-a0a3-201f6036816e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:50.727617 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 582/1157] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:15:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:50.733372 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae2e97-7f75-4dcd-918d-4fee1ead1b6b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:50.831020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae2e97-7f75-4dcd-918d-4fee1ead1b6b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:50.852393 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:50.863335 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35508685-a18b-4457-8fb1-b65ec22e21d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:50.864198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35508685-a18b-4457-8fb1-b65ec22e21d5 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:50.864477 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35508685-a18b-4457-8fb1-b65ec22e21d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:50.864748 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35508685-a18b-4457-8fb1-b65ec22e21d5 None admin] PATH_INFO: `/v3/users/9d738ac19de4442fa2ec7a1cbc63f270` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:50.865913 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35508685-a18b-4457-8fb1-b65ec22e21d5 None admin] RBAC: Authorizing `identity:get_user(user_id=9d738ac19de4442fa2ec7a1cbc63f270)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:50.867385 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35508685-a18b-4457-8fb1-b65ec22e21d5 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:50.868984 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 576/1158] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:15:50 2022] GET /identity/v3/users/9d738ac19de4442fa2ec7a1cbc63f270 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:50.875503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13940dd3-eaa7-49a6-a0a3-201f6036816e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:50.974444 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13940dd3-eaa7-49a6-a0a3-201f6036816e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:51.006540 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e902561-36e9-4371-bc75-c3e820f1df72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:51.007181 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e902561-36e9-4371-bc75-c3e820f1df72 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:51.007650 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e902561-36e9-4371-bc75-c3e820f1df72 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:51.007931 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e902561-36e9-4371-bc75-c3e820f1df72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:51.037530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:51.073130 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e902561-36e9-4371-bc75-c3e820f1df72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:51.074584 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e902561-36e9-4371-bc75-c3e820f1df72 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:51.175552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 583/1159] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:15:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 10:15:51.181644 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35508685-a18b-4457-8fb1-b65ec22e21d5 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:51.279632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35508685-a18b-4457-8fb1-b65ec22e21d5 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:51.312692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec4beb9-dd8e-411c-a96d-92218cab9c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:51.313544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4beb9-dd8e-411c-a96d-92218cab9c8b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:51.313832 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4beb9-dd8e-411c-a96d-92218cab9c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:51.314110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec4beb9-dd8e-411c-a96d-92218cab9c8b None admin] PATH_INFO: `/v3/projects/27d92635bfcb446d85e3eab74791598d` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:51.315339 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4beb9-dd8e-411c-a96d-92218cab9c8b None admin] RBAC: Authorizing `identity:get_project(project_id=27d92635bfcb446d85e3eab74791598d)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:51.316759 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec4beb9-dd8e-411c-a96d-92218cab9c8b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:51.318436 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 577/1160] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:15:51 2022] GET /identity/v3/projects/27d92635bfcb446d85e3eab74791598d => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:51.324679 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e902561-36e9-4371-bc75-c3e820f1df72 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:51.384313 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:51.422152 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e902561-36e9-4371-bc75-c3e820f1df72 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:51.455523 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8244995-1cde-4c88-8c30-3705a29d375d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:51.456093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8244995-1cde-4c88-8c30-3705a29d375d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:51.456377 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8244995-1cde-4c88-8c30-3705a29d375d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:51.456653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8244995-1cde-4c88-8c30-3705a29d375d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:51.457479 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8244995-1cde-4c88-8c30-3705a29d375d None admin] RBAC: Adding query filter params (role.id=e63bda042e104a4a93fba15ffcfe1183, scope.project.id=27d92635bfcb446d85e3eab74791598d, user.id=9d738ac19de4442fa2ec7a1cbc63f270) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:51.457789 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8244995-1cde-4c88-8c30-3705a29d375d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:51.459224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8244995-1cde-4c88-8c30-3705a29d375d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:51.464915 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 584/1161] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:15:51 2022] GET /identity/v3/role_assignments?user.id=9d738ac19de4442fa2ec7a1cbc63f270&scope.project.id=27d92635bfcb446d85e3eab74791598d&role.id=e63bda042e104a4a93fba15ffcfe1183 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:53.500883 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43ce86-c403-417f-a651-57fd963486f1 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:53.501419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43ce86-c403-417f-a651-57fd963486f1 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:53.501419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e43ce86-c403-417f-a651-57fd963486f1 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:53.503192 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 578/1162] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:15:53.508173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f784a8-c0c9-4bfc-8b9f-c009b264a690 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:53.508617 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f784a8-c0c9-4bfc-8b9f-c009b264a690 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:53.508767 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f784a8-c0c9-4bfc-8b9f-c009b264a690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:53.526260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-55f784a8-c0c9-4bfc-8b9f-c009b264a690 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:53.527479 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-55f784a8-c0c9-4bfc-8b9f-c009b264a690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:53.592625 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 585/1163] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:15:53.614775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43ce86-c403-417f-a651-57fd963486f1 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:53.674390 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:53.715096 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e43ce86-c403-417f-a651-57fd963486f1 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:53.748817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a10f5-92e2-487c-a518-d0df7307df02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:53.749471 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a10f5-92e2-487c-a518-d0df7307df02 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:53.749756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a10f5-92e2-487c-a518-d0df7307df02 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:53.750032 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07a10f5-92e2-487c-a518-d0df7307df02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:53.817981 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a10f5-92e2-487c-a518-d0df7307df02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:53.819300 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07a10f5-92e2-487c-a518-d0df7307df02 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:53.875837 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:53.917453 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 579/1164] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:15:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 10:15:53.923774 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f784a8-c0c9-4bfc-8b9f-c009b264a690 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:53.979996 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:54.020515 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f784a8-c0c9-4bfc-8b9f-c009b264a690 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:54.052637 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a8965d-9318-4ab3-85c6-bc1d6afa7d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:54.053429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8965d-9318-4ab3-85c6-bc1d6afa7d6d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:54.053717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8965d-9318-4ab3-85c6-bc1d6afa7d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:54.053997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a8965d-9318-4ab3-85c6-bc1d6afa7d6d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:54.060941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8965d-9318-4ab3-85c6-bc1d6afa7d6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:54.062397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a8965d-9318-4ab3-85c6-bc1d6afa7d6d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:54.077780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-75a8965d-9318-4ab3-85c6-bc1d6afa7d6d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 30 10:15:54.077780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:54.077780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 30 10:15:54.077780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 30 10:15:54.077780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:15:54.077780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:15:54.077780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:15:54.077780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:15:54.077780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 30 10:15:54.077780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:54.077780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:15:54.077780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:54.077780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:15:54.077780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:15:54.077780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:15:54.077780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:15:54.077780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:15:54.079018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:15:54.079018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:15:54.079018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:15:54.079018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:15:54.079018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:15:54.079018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:15:54.079018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:15:54.079018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:15:54.079018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:15:54.079018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:15:54.079018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:15:54.079018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:15:54.079018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:15:54.079018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:15:54.079018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:15:54.079018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:15:54.080112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:15:54.080112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:15:54.080112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:15:54.080112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:15:54.080112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:15:54.080112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:15:54.080112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:15:54.080112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:15:54.080112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:15:54.080112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:15:54.080112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 30 10:15:54.080112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:15:54.080112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 30 10:15:54.080112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:15:54.080112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 586/1165] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jun 30 10:15:53 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:15:54.085035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a10f5-92e2-487c-a518-d0df7307df02 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:54.182498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07a10f5-92e2-487c-a518-d0df7307df02 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:54.215836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbad1a0-1ba2-4e68-a058-c842c8c89355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:54.216493 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbad1a0-1ba2-4e68-a058-c842c8c89355 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:54.216830 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbad1a0-1ba2-4e68-a058-c842c8c89355 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:54.217187 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbad1a0-1ba2-4e68-a058-c842c8c89355 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:54.217907 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbad1a0-1ba2-4e68-a058-c842c8c89355 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:15:54.218214 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbad1a0-1ba2-4e68-a058-c842c8c89355 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:54.219613 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbad1a0-1ba2-4e68-a058-c842c8c89355 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:54.226508 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 580/1166] 10.222.0.47 () {56 vars in 1115 bytes} [Thu Jun 30 10:15:54 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:56.234443 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b319c-d5ee-47e9-8616-6fd8b10d6842 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:56.234988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b319c-d5ee-47e9-8616-6fd8b10d6842 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:56.234988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b319c-d5ee-47e9-8616-6fd8b10d6842 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:56.236578 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 587/1167] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:15:56.241733 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981ff95-13fa-485b-8f8c-ccb245791564 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:56.242135 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981ff95-13fa-485b-8f8c-ccb245791564 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:56.242295 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981ff95-13fa-485b-8f8c-ccb245791564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:56.258765 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-a981ff95-13fa-485b-8f8c-ccb245791564 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:56.259980 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-a981ff95-13fa-485b-8f8c-ccb245791564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:56.276122 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:56.326685 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 581/1168] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:15:56.350339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b319c-d5ee-47e9-8616-6fd8b10d6842 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:56.449407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b319c-d5ee-47e9-8616-6fd8b10d6842 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:56.466758 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:56.483115 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6079b8dc-c997-4a8c-ad8e-55afd75b1734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:56.483838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079b8dc-c997-4a8c-ad8e-55afd75b1734 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:56.484124 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079b8dc-c997-4a8c-ad8e-55afd75b1734 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:56.484401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6079b8dc-c997-4a8c-ad8e-55afd75b1734 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:56.485199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6079b8dc-c997-4a8c-ad8e-55afd75b1734 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:56.486524 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6079b8dc-c997-4a8c-ad8e-55afd75b1734 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:56.559886 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 588/1169] 10.222.0.47 () {60 vars in 1138 bytes} [Thu Jun 30 10:15:56 2022] POST /identity/v3/domains => generated 267 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:15:58.603756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd7c7d-0225-4643-9d67-fabbcab51210 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:58.604339 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd7c7d-0225-4643-9d67-fabbcab51210 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:58.604339 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dd7c7d-0225-4643-9d67-fabbcab51210 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:58.605685 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 582/1170] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:15:58.610858 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbbb2a-c807-4cd9-b5ba-b712ef1d49a6 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:58.611323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbbb2a-c807-4cd9-b5ba-b712ef1d49a6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:58.611461 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbbb2a-c807-4cd9-b5ba-b712ef1d49a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:58.628551 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a8bbbb2a-c807-4cd9-b5ba-b712ef1d49a6 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:15:58.629726 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a8bbbb2a-c807-4cd9-b5ba-b712ef1d49a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:15:58.694232 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 589/1171] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:15:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:15:58.718797 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd7c7d-0225-4643-9d67-fabbcab51210 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:58.818929 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dd7c7d-0225-4643-9d67-fabbcab51210 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:58.865665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5978bc40-5eaa-459f-b7de-44104d355eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:58.867071 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978bc40-5eaa-459f-b7de-44104d355eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:58.867385 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978bc40-5eaa-459f-b7de-44104d355eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:58.867753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5978bc40-5eaa-459f-b7de-44104d355eb7 None admin] PATH_INFO: `/v3/domains/cdf2f10c8dc645b8a79acaca67431a4e` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:58.880842 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5978bc40-5eaa-459f-b7de-44104d355eb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=cdf2f10c8dc645b8a79acaca67431a4e)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:58.881375 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5978bc40-5eaa-459f-b7de-44104d355eb7 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:58.883003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 583/1172] 10.222.0.47 () {56 vars in 1183 bytes} [Thu Jun 30 10:15:58 2022] GET /identity/v3/domains/cdf2f10c8dc645b8a79acaca67431a4e => generated 267 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:15:58.889596 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bbbb2a-c807-4cd9-b5ba-b712ef1d49a6 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:15:58.986504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bbbb2a-c807-4cd9-b5ba-b712ef1d49a6 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:15:58.995325 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:15:59.019010 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4901004-3b0c-400f-bf58-bd4e57a64797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:15:59.019698 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4901004-3b0c-400f-bf58-bd4e57a64797 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:15:59.020006 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4901004-3b0c-400f-bf58-bd4e57a64797 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:15:59.020283 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4901004-3b0c-400f-bf58-bd4e57a64797 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:15:59.020985 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4901004-3b0c-400f-bf58-bd4e57a64797 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:15:59.022686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4901004-3b0c-400f-bf58-bd4e57a64797 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:15:59.114733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 590/1173] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jun 30 10:15:58 2022] POST /identity/v3/projects => generated 361 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:16:01.187193 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abc530-aae8-479a-81d0-b8fa2fa9e735 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:01.187783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abc530-aae8-479a-81d0-b8fa2fa9e735 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:01.187783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abc530-aae8-479a-81d0-b8fa2fa9e735 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:01.189124 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 584/1174] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:16:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:16:01.194324 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ca36e-89ea-4ea0-b07d-0b73bb32340d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:01.194730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ca36e-89ea-4ea0-b07d-0b73bb32340d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:01.194891 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ca36e-89ea-4ea0-b07d-0b73bb32340d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:01.212088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6e1ca36e-89ea-4ea0-b07d-0b73bb32340d None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:16:01.213263 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6e1ca36e-89ea-4ea0-b07d-0b73bb32340d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:16:01.277622 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 591/1175] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:16:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:16:01.303104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43abc530-aae8-479a-81d0-b8fa2fa9e735 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:16:01.318441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:16:01.403593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43abc530-aae8-479a-81d0-b8fa2fa9e735 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:16:01.435573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8393845-c393-44e5-aa42-c85f6570305e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:16:01.436512 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8393845-c393-44e5-aa42-c85f6570305e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:01.436793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8393845-c393-44e5-aa42-c85f6570305e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:01.437071 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8393845-c393-44e5-aa42-c85f6570305e None admin] PATH_INFO: `/v3/domains/cdf2f10c8dc645b8a79acaca67431a4e` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:01.438319 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8393845-c393-44e5-aa42-c85f6570305e None admin] RBAC: Authorizing `identity:get_domain(domain_id=cdf2f10c8dc645b8a79acaca67431a4e)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:16:01.439729 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8393845-c393-44e5-aa42-c85f6570305e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:16:01.441329 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 585/1176] 10.222.0.47 () {56 vars in 1183 bytes} [Thu Jun 30 10:16:01 2022] GET /identity/v3/domains/cdf2f10c8dc645b8a79acaca67431a4e => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:16:01.447110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ca36e-89ea-4ea0-b07d-0b73bb32340d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:16:01.534835 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:16:01.543909 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ca36e-89ea-4ea0-b07d-0b73bb32340d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:16:01.576808 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5792a62-dd3a-4be4-a0b9-4c0c76396747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:16:01.577466 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5792a62-dd3a-4be4-a0b9-4c0c76396747 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:01.577750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5792a62-dd3a-4be4-a0b9-4c0c76396747 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:01.578027 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5792a62-dd3a-4be4-a0b9-4c0c76396747 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:01.578732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5792a62-dd3a-4be4-a0b9-4c0c76396747 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:16:01.580191 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5792a62-dd3a-4be4-a0b9-4c0c76396747 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:16:01.676889 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 592/1177] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jun 30 10:16:01 2022] POST /identity/v3/users => generated 311 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:16:03.759121 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba55bc-1007-4f6c-b41a-a4a52e706b32 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:03.759792 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba55bc-1007-4f6c-b41a-a4a52e706b32 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:03.759792 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba55bc-1007-4f6c-b41a-a4a52e706b32 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:03.760969 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 586/1178] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:16:03.766999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ec79b-4670-487f-8c96-00b4e2e115f1 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:03.767455 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ec79b-4670-487f-8c96-00b4e2e115f1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:03.767595 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078ec79b-4670-487f-8c96-00b4e2e115f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:03.784555 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-078ec79b-4670-487f-8c96-00b4e2e115f1 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:16:03.785738 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-078ec79b-4670-487f-8c96-00b4e2e115f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:16:03.850189 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 593/1179] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:16:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:16:03.873845 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba55bc-1007-4f6c-b41a-a4a52e706b32 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:16:03.961014 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:16:03.980116 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba55bc-1007-4f6c-b41a-a4a52e706b32 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:16:04.015367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6098a852-0383-4b78-8159-7c3396e4c9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:16:04.016086 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098a852-0383-4b78-8159-7c3396e4c9f7 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:04.016369 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098a852-0383-4b78-8159-7c3396e4c9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:04.016647 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6098a852-0383-4b78-8159-7c3396e4c9f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:04.026783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6098a852-0383-4b78-8159-7c3396e4c9f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:16:04.034510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-6098a852-0383-4b78-8159-7c3396e4c9f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:16:04.034510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:16:04.034510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:16:04.034510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:16:04.034510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:16:04.034510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:16:04.034510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:16:04.034510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:16:04.034510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:16:04.034510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:16:04.034510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:16:04.034510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:16:04.034510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:16:04.034510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:16:04.034510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:16:04.034510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:16:04.035814 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:16:04.035814 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:16:04.035814 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:16:04.035814 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:16:04.035814 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:16:04.035814 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:16:04.035814 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:16:04.035814 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:16:04.035814 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:16:04.035814 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:16:04.035814 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:16:04.035814 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:16:04.035814 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:16:04.035814 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:16:04.035814 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:16:04.035814 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:16:04.036742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:16:04.036742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:16:04.036742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:16:04.036742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:16:04.036742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:16:04.036742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:16:04.036742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:16:04.036742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:16:04.036742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:16:04.036742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:16:04.036742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:16:04.036742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:16:04.036742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 587/1180] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:16:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:16:04.041762 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ec79b-4670-487f-8c96-00b4e2e115f1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:16:04.117188 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:16:04.140419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078ec79b-4670-487f-8c96-00b4e2e115f1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:16:04.174699 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674cfe6-7cf2-46e6-a1f0-c5a66c1150d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:16:04.175384 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674cfe6-7cf2-46e6-a1f0-c5a66c1150d9 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:04.175669 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674cfe6-7cf2-46e6-a1f0-c5a66c1150d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:04.175946 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674cfe6-7cf2-46e6-a1f0-c5a66c1150d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:04.176693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674cfe6-7cf2-46e6-a1f0-c5a66c1150d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:16:04.176993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674cfe6-7cf2-46e6-a1f0-c5a66c1150d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:16:04.178367 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674cfe6-7cf2-46e6-a1f0-c5a66c1150d9 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:16:04.191372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 594/1181] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:16:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:16:04.193023 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6098a852-0383-4b78-8159-7c3396e4c9f7 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:16:04.291197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6098a852-0383-4b78-8159-7c3396e4c9f7 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:16:04.326656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8076b-6ac9-4c00-aaa7-873284c9fae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:16:04.327543 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8076b-6ac9-4c00-aaa7-873284c9fae9 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:04.327831 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8076b-6ac9-4c00-aaa7-873284c9fae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:04.328107 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f8076b-6ac9-4c00-aaa7-873284c9fae9 None admin] PATH_INFO: `/v3/users/4cabe65097434cc29e140ddf14588664` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:04.338081 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:16:04.341879 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8076b-6ac9-4c00-aaa7-873284c9fae9 None admin] RBAC: Authorizing `identity:get_user(user_id=4cabe65097434cc29e140ddf14588664)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:16:04.343466 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f8076b-6ac9-4c00-aaa7-873284c9fae9 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:16:04.345113 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 588/1182] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:16:04 2022] GET /identity/v3/users/4cabe65097434cc29e140ddf14588664 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:16:04.350824 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674cfe6-7cf2-46e6-a1f0-c5a66c1150d9 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:16:04.447927 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674cfe6-7cf2-46e6-a1f0-c5a66c1150d9 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:16:04.480266 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4803028-3692-4836-b2e1-65044aef88ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:16:04.480912 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4803028-3692-4836-b2e1-65044aef88ec None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:04.481229 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4803028-3692-4836-b2e1-65044aef88ec None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:04.481511 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4803028-3692-4836-b2e1-65044aef88ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:04.490048 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:16:04.547719 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4803028-3692-4836-b2e1-65044aef88ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:16:04.549127 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4803028-3692-4836-b2e1-65044aef88ec None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:16:04.642776 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 595/1183] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:16:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 10:16:04.648857 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8076b-6ac9-4c00-aaa7-873284c9fae9 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:16:04.745094 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f8076b-6ac9-4c00-aaa7-873284c9fae9 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:16:04.777953 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3711207-2de2-4329-8b35-30cab73ba727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:16:04.778997 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3711207-2de2-4329-8b35-30cab73ba727 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:04.779410 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3711207-2de2-4329-8b35-30cab73ba727 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:04.779606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3711207-2de2-4329-8b35-30cab73ba727 None admin] PATH_INFO: `/v3/projects/e9a9fe28853d47e5ba907d7f7c93d2a8` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:04.780790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3711207-2de2-4329-8b35-30cab73ba727 None admin] RBAC: Authorizing `identity:get_project(project_id=e9a9fe28853d47e5ba907d7f7c93d2a8)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:16:04.782214 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3711207-2de2-4329-8b35-30cab73ba727 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:16:04.783925 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 589/1184] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:16:04 2022] GET /identity/v3/projects/e9a9fe28853d47e5ba907d7f7c93d2a8 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:16:04.789792 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4803028-3692-4836-b2e1-65044aef88ec None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:16:04.829337 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:16:04.886518 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4803028-3692-4836-b2e1-65044aef88ec None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:16:04.919206 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dd6d30-f826-45e6-bc94-4982a6eda1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:16:04.920991 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd6d30-f826-45e6-bc94-4982a6eda1dd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:04.921271 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd6d30-f826-45e6-bc94-4982a6eda1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:04.921542 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dd6d30-f826-45e6-bc94-4982a6eda1dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:04.922360 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd6d30-f826-45e6-bc94-4982a6eda1dd None admin] RBAC: Adding query filter params (role.id=e63bda042e104a4a93fba15ffcfe1183, scope.project.id=e9a9fe28853d47e5ba907d7f7c93d2a8, user.id=4cabe65097434cc29e140ddf14588664) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:16:04.922670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd6d30-f826-45e6-bc94-4982a6eda1dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:16:04.924120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dd6d30-f826-45e6-bc94-4982a6eda1dd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:16:04.929346 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 596/1185] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:16:04 2022] GET /identity/v3/role_assignments?user.id=4cabe65097434cc29e140ddf14588664&scope.project.id=e9a9fe28853d47e5ba907d7f7c93d2a8&role.id=e63bda042e104a4a93fba15ffcfe1183 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:16:06.957032 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa76fd-5259-42b0-939d-edf0035bb69b None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:06.957580 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa76fd-5259-42b0-939d-edf0035bb69b None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:06.957580 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa76fd-5259-42b0-939d-edf0035bb69b None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:06.959252 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 590/1186] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:16:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:16:06.964557 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdf1fa-5ebc-4782-9719-74f08cc32b09 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:06.964945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdf1fa-5ebc-4782-9719-74f08cc32b09 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:06.965121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cdf1fa-5ebc-4782-9719-74f08cc32b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:06.984706 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e0cdf1fa-5ebc-4782-9719-74f08cc32b09 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:16:06.985915 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e0cdf1fa-5ebc-4782-9719-74f08cc32b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:16:07.050619 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 597/1187] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:16:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:16:07.073581 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa76fd-5259-42b0-939d-edf0035bb69b None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:16:07.140098 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:16:07.174150 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa76fd-5259-42b0-939d-edf0035bb69b None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:16:07.209093 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50fe6c-6568-44f2-9bc1-458aed344941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:16:07.209800 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50fe6c-6568-44f2-9bc1-458aed344941 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:07.210079 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50fe6c-6568-44f2-9bc1-458aed344941 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:07.210348 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f50fe6c-6568-44f2-9bc1-458aed344941 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:07.220452 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f50fe6c-6568-44f2-9bc1-458aed344941 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:16:07.228004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-4f50fe6c-6568-44f2-9bc1-458aed344941 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:16:07.228004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:16:07.228004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:16:07.228004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:16:07.228004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:16:07.228004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:16:07.228004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:16:07.228004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:16:07.228004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:16:07.228004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:16:07.228004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:16:07.228004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:16:07.228004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:16:07.228004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:16:07.228004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:16:07.228004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:16:07.229108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:16:07.229108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:16:07.229108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:16:07.229108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:16:07.229108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:16:07.229108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:16:07.229108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:16:07.229108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:16:07.229108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:16:07.229108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:16:07.229108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:16:07.229108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:16:07.229108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:16:07.229108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:16:07.229108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:16:07.229108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:16:07.230184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:16:07.230184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:16:07.230184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:16:07.230184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:16:07.230184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:16:07.230184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:16:07.230184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:16:07.230184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:16:07.230184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:16:07.230184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:16:07.230184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:16:07.230184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:16:07.230184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 591/1188] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:16:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:16:07.236196 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdf1fa-5ebc-4782-9719-74f08cc32b09 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:16:07.265626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:16:07.332278 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cdf1fa-5ebc-4782-9719-74f08cc32b09 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:16:07.365375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501dd53-bc4b-4915-b98c-2aaf64f3def8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:16:07.366032 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501dd53-bc4b-4915-b98c-2aaf64f3def8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:07.366318 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501dd53-bc4b-4915-b98c-2aaf64f3def8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:07.366597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501dd53-bc4b-4915-b98c-2aaf64f3def8 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:07.367369 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501dd53-bc4b-4915-b98c-2aaf64f3def8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:16:07.367682 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501dd53-bc4b-4915-b98c-2aaf64f3def8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:16:07.369057 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501dd53-bc4b-4915-b98c-2aaf64f3def8 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:16:07.376845 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 598/1189] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:16:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:16:07.382859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50fe6c-6568-44f2-9bc1-458aed344941 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:16:07.481758 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f50fe6c-6568-44f2-9bc1-458aed344941 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:16:07.506531 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:16:07.515533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399b087-b86c-469e-ab0e-98068cf70347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:16:07.522302 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399b087-b86c-469e-ab0e-98068cf70347 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:07.522302 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399b087-b86c-469e-ab0e-98068cf70347 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:07.522302 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6399b087-b86c-469e-ab0e-98068cf70347 None admin] PATH_INFO: `/v3/users/4cabe65097434cc29e140ddf14588664` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:07.522302 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399b087-b86c-469e-ab0e-98068cf70347 None admin] RBAC: Authorizing `identity:get_user(user_id=4cabe65097434cc29e140ddf14588664)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:16:07.522302 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6399b087-b86c-469e-ab0e-98068cf70347 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:16:07.522302 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 592/1190] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:16:07 2022] GET /identity/v3/users/4cabe65097434cc29e140ddf14588664 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:16:07.527301 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501dd53-bc4b-4915-b98c-2aaf64f3def8 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:16:07.616970 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:16:07.623772 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501dd53-bc4b-4915-b98c-2aaf64f3def8 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:16:07.656720 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416fb4ec-5edc-486e-a137-247053cd39cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:16:07.657562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fb4ec-5edc-486e-a137-247053cd39cc None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:07.657841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fb4ec-5edc-486e-a137-247053cd39cc None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:07.658111 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416fb4ec-5edc-486e-a137-247053cd39cc None admin] PATH_INFO: `/v3/projects/e9a9fe28853d47e5ba907d7f7c93d2a8` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:07.659286 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416fb4ec-5edc-486e-a137-247053cd39cc None admin] RBAC: Authorizing `identity:get_project(project_id=e9a9fe28853d47e5ba907d7f7c93d2a8)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:16:07.660711 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416fb4ec-5edc-486e-a137-247053cd39cc None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:16:07.662369 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 599/1191] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:16:07 2022] GET /identity/v3/projects/e9a9fe28853d47e5ba907d7f7c93d2a8 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:16:07.668094 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399b087-b86c-469e-ab0e-98068cf70347 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:16:07.765474 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6399b087-b86c-469e-ab0e-98068cf70347 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:16:07.798115 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4b2c85-1d26-4fc2-9ca4-9e7f0092ccda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:16:07.798987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b2c85-1d26-4fc2-9ca4-9e7f0092ccda None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:07.799302 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b2c85-1d26-4fc2-9ca4-9e7f0092ccda None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:07.799637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4b2c85-1d26-4fc2-9ca4-9e7f0092ccda None admin] PATH_INFO: `/v3/projects/e9a9fe28853d47e5ba907d7f7c93d2a8/users/4cabe65097434cc29e140ddf14588664/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:07.801547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b2c85-1d26-4fc2-9ca4-9e7f0092ccda None admin] RBAC: Authorizing `identity:create_grant(project_id=e9a9fe28853d47e5ba907d7f7c93d2a8, user_id=4cabe65097434cc29e140ddf14588664, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:16:07.802968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4b2c85-1d26-4fc2-9ca4-9e7f0092ccda None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:16:07.901379 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 593/1192] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jun 30 10:16:07 2022] PUT /identity/v3/projects/e9a9fe28853d47e5ba907d7f7c93d2a8/users/4cabe65097434cc29e140ddf14588664/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:16:09.901528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdd642-8850-45a5-a10b-7a83fe3429cd None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:09.902074 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdd642-8850-45a5-a10b-7a83fe3429cd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:09.902074 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbdd642-8850-45a5-a10b-7a83fe3429cd None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:09.903446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 600/1193] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:16:09.909250 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae221bb-104c-4838-8006-fbad917de467 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:09.909659 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae221bb-104c-4838-8006-fbad917de467 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:09.909847 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae221bb-104c-4838-8006-fbad917de467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:09.926999 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-aae221bb-104c-4838-8006-fbad917de467 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:16:09.928254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-aae221bb-104c-4838-8006-fbad917de467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:16:09.978628 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:16:09.995810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 594/1194] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:16:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:16:10.020120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdd642-8850-45a5-a10b-7a83fe3429cd None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:16:10.119280 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbdd642-8850-45a5-a10b-7a83fe3429cd None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:16:10.153493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df118e-3569-4c30-ac2d-b0e0b05a15e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:16:10.154208 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df118e-3569-4c30-ac2d-b0e0b05a15e8 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:10.154496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df118e-3569-4c30-ac2d-b0e0b05a15e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:10.154767 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df118e-3569-4c30-ac2d-b0e0b05a15e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:10.164902 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33df118e-3569-4c30-ac2d-b0e0b05a15e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:16:10.173031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-33df118e-3569-4c30-ac2d-b0e0b05a15e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:16:10.173031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:16:10.173031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:16:10.173031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:16:10.173031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:16:10.173031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:16:10.173031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:16:10.173031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:16:10.173031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:16:10.173031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:16:10.173031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:16:10.173031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:16:10.173031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:16:10.173031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:16:10.173031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:16:10.173031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:16:10.174154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:16:10.174154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:16:10.174154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:16:10.174154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:16:10.174154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:16:10.174154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:16:10.174154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:16:10.174154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:16:10.174154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:16:10.174154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:16:10.174154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:16:10.174154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:16:10.174154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:16:10.174154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:16:10.174154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:16:10.174154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:16:10.175390 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:16:10.175390 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:16:10.175390 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:16:10.175390 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:16:10.175390 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:16:10.175390 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:16:10.175390 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:16:10.175390 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:16:10.175390 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:16:10.175390 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:16:10.175390 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:16:10.175390 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:16:10.175390 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 601/1195] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:16:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:16:10.180245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae221bb-104c-4838-8006-fbad917de467 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:16:10.277875 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae221bb-104c-4838-8006-fbad917de467 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:16:10.310732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c2840-de75-4fba-849b-34e0e1a585ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:16:10.311431 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c2840-de75-4fba-849b-34e0e1a585ec None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:10.311720 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c2840-de75-4fba-849b-34e0e1a585ec None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:10.312003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c2840-de75-4fba-849b-34e0e1a585ec None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:10.312756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4c2840-de75-4fba-849b-34e0e1a585ec None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:16:10.313067 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4c2840-de75-4fba-849b-34e0e1a585ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:16:10.314442 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4c2840-de75-4fba-849b-34e0e1a585ec None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:16:10.322431 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 595/1196] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:16:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:16:10.328691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df118e-3569-4c30-ac2d-b0e0b05a15e8 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:16:10.353109 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:16:10.425324 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33df118e-3569-4c30-ac2d-b0e0b05a15e8 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:16:10.458526 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0ba72-5d9f-46d3-8af2-288dc87fa262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:16:10.459412 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0ba72-5d9f-46d3-8af2-288dc87fa262 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:10.459699 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0ba72-5d9f-46d3-8af2-288dc87fa262 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:10.459978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0ba72-5d9f-46d3-8af2-288dc87fa262 None admin] PATH_INFO: `/v3/users/4cabe65097434cc29e140ddf14588664` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:10.461158 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0ba72-5d9f-46d3-8af2-288dc87fa262 None admin] RBAC: Authorizing `identity:get_user(user_id=4cabe65097434cc29e140ddf14588664)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:16:10.462596 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0ba72-5d9f-46d3-8af2-288dc87fa262 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:16:10.464246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 602/1197] 10.222.0.47 () {56 vars in 1177 bytes} [Thu Jun 30 10:16:10 2022] GET /identity/v3/users/4cabe65097434cc29e140ddf14588664 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:16:10.470173 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c2840-de75-4fba-849b-34e0e1a585ec None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:16:10.516517 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:16:10.568232 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c2840-de75-4fba-849b-34e0e1a585ec None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:16:10.601029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f04fa-1b54-45b6-9694-ac48b0af5188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:16:10.601681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f04fa-1b54-45b6-9694-ac48b0af5188 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:10.601965 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f04fa-1b54-45b6-9694-ac48b0af5188 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:10.602241 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532f04fa-1b54-45b6-9694-ac48b0af5188 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:10.667677 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f04fa-1b54-45b6-9694-ac48b0af5188 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:16:10.669202 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532f04fa-1b54-45b6-9694-ac48b0af5188 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:16:10.713439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:16:10.765054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 596/1198] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:16:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 10:16:10.771025 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0ba72-5d9f-46d3-8af2-288dc87fa262 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:16:10.858014 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:16:10.867060 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0ba72-5d9f-46d3-8af2-288dc87fa262 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:16:10.900300 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1098f682-a9e6-4a31-b375-c03a8c0c7fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:16:10.901151 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098f682-a9e6-4a31-b375-c03a8c0c7fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:10.901440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098f682-a9e6-4a31-b375-c03a8c0c7fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:10.901720 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098f682-a9e6-4a31-b375-c03a8c0c7fc0 None admin] PATH_INFO: `/v3/projects/e9a9fe28853d47e5ba907d7f7c93d2a8` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:10.902902 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1098f682-a9e6-4a31-b375-c03a8c0c7fc0 None admin] RBAC: Authorizing `identity:get_project(project_id=e9a9fe28853d47e5ba907d7f7c93d2a8)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:16:10.904435 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1098f682-a9e6-4a31-b375-c03a8c0c7fc0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:16:10.906130 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 603/1199] 10.222.0.47 () {56 vars in 1186 bytes} [Thu Jun 30 10:16:10 2022] GET /identity/v3/projects/e9a9fe28853d47e5ba907d7f7c93d2a8 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:16:10.912152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f04fa-1b54-45b6-9694-ac48b0af5188 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:16:11.010332 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532f04fa-1b54-45b6-9694-ac48b0af5188 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:16:11.043404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c5776-6bc5-4db1-8c86-936ef5fd04fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:16:11.044647 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c5776-6bc5-4db1-8c86-936ef5fd04fc None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:16:11.045410 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c5776-6bc5-4db1-8c86-936ef5fd04fc None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:16:11.045689 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c5776-6bc5-4db1-8c86-936ef5fd04fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:16:11.046512 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c5776-6bc5-4db1-8c86-936ef5fd04fc None admin] RBAC: Adding query filter params (role.id=e63bda042e104a4a93fba15ffcfe1183, scope.project.id=e9a9fe28853d47e5ba907d7f7c93d2a8, user.id=4cabe65097434cc29e140ddf14588664) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:16:11.046827 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c5776-6bc5-4db1-8c86-936ef5fd04fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:16:11.048275 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c5776-6bc5-4db1-8c86-936ef5fd04fc None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:16:11.053882 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 597/1200] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:16:10 2022] GET /identity/v3/role_assignments?user.id=4cabe65097434cc29e140ddf14588664&scope.project.id=e9a9fe28853d47e5ba907d7f7c93d2a8&role.id=e63bda042e104a4a93fba15ffcfe1183 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:16:20.896037 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=21 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:16:21.051278 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=95 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:28.515396 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea49b4-6bab-4d95-bc03-de54c3c00fe6 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:28.515845 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea49b4-6bab-4d95-bc03-de54c3c00fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:28.515952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea49b4-6bab-4d95-bc03-de54c3c00fe6 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:28.517325 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 604/1201] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:17:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:17:28.522836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc5e71-9114-4bb1-ac7c-10d44c70b023 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:28.523230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc5e71-9114-4bb1-ac7c-10d44c70b023 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:28.523430 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdc5e71-9114-4bb1-ac7c-10d44c70b023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:28.540557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-3cdc5e71-9114-4bb1-ac7c-10d44c70b023 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:17:28.541755 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-3cdc5e71-9114-4bb1-ac7c-10d44c70b023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:17:28.613697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 598/1202] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:17:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:17:28.639294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea49b4-6bab-4d95-bc03-de54c3c00fe6 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:28.742900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea49b4-6bab-4d95-bc03-de54c3c00fe6 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:28.776406 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ef1ec-c6ff-4988-bd4c-d43eb92bd1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:28.777100 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ef1ec-c6ff-4988-bd4c-d43eb92bd1fd None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:28.777382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ef1ec-c6ff-4988-bd4c-d43eb92bd1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:28.777655 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ef1ec-c6ff-4988-bd4c-d43eb92bd1fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:28.787907 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ef1ec-c6ff-4988-bd4c-d43eb92bd1fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:28.789305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ef1ec-c6ff-4988-bd4c-d43eb92bd1fd None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:28.807685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-034ef1ec-c6ff-4988-bd4c-d43eb92bd1fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:17:28.807685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:28.807685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:28.807685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:28.807685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:28.807685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:28.807685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:28.807685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:28.807685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:28.807685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:28.807685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:28.807685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:28.807685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:17:28.807685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:17:28.807685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:17:28.808818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:17:28.808818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:28.808818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:28.808818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:28.808818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:28.808818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:28.808818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:28.808818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:28.808818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:28.808818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:28.808818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:28.808818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:28.808818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:28.808818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:28.808818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:28.808818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:28.808818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:28.814815 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:17:28.814815 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:17:28.814815 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:17:28.814815 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:17:28.814815 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 605/1203] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:17:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:28.815197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc5e71-9114-4bb1-ac7c-10d44c70b023 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:28.912378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdc5e71-9114-4bb1-ac7c-10d44c70b023 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:28.939595 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:28.945319 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d5daa-1367-4ca3-a446-4193ba9e83ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:28.945977 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d5daa-1367-4ca3-a446-4193ba9e83ae None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:28.946260 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d5daa-1367-4ca3-a446-4193ba9e83ae None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:28.946534 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d5daa-1367-4ca3-a446-4193ba9e83ae None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:28.947249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d5daa-1367-4ca3-a446-4193ba9e83ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:28.947575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d5daa-1367-4ca3-a446-4193ba9e83ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:28.949080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d5daa-1367-4ca3-a446-4193ba9e83ae None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:28.960703 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 599/1204] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:17:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:28.966352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ef1ec-c6ff-4988-bd4c-d43eb92bd1fd None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:29.017512 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:29.064068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ef1ec-c6ff-4988-bd4c-d43eb92bd1fd None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:29.096059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5191c85d-e1d5-4690-8952-589ac4bf5caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:29.096719 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191c85d-e1d5-4690-8952-589ac4bf5caf None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:29.097001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191c85d-e1d5-4690-8952-589ac4bf5caf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:29.097274 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5191c85d-e1d5-4690-8952-589ac4bf5caf None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:29.097976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5191c85d-e1d5-4690-8952-589ac4bf5caf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:29.100335 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5191c85d-e1d5-4690-8952-589ac4bf5caf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:29.173478 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 606/1205] 10.222.0.47 () {60 vars in 1132 bytes} [Thu Jun 30 10:17:28 2022] POST /identity/v3/users => generated 251 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:17:31.234565 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869ba00-f3f8-4c7c-bcb3-702ca2eff013 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:31.235159 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869ba00-f3f8-4c7c-bcb3-702ca2eff013 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:31.235159 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869ba00-f3f8-4c7c-bcb3-702ca2eff013 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:31.236525 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 600/1206] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:17:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:17:31.242215 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd2b1b-de22-40b9-9886-c6f0cdb89c93 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:31.242668 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd2b1b-de22-40b9-9886-c6f0cdb89c93 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:31.242773 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd2b1b-de22-40b9-9886-c6f0cdb89c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:31.260337 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-64cd2b1b-de22-40b9-9886-c6f0cdb89c93 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:17:31.261524 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-64cd2b1b-de22-40b9-9886-c6f0cdb89c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:17:31.348115 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 607/1207] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:17:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:17:31.372304 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869ba00-f3f8-4c7c-bcb3-702ca2eff013 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:31.476387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869ba00-f3f8-4c7c-bcb3-702ca2eff013 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:31.509999 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768080af-30a1-463c-9225-ff6438bbb704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:31.510909 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768080af-30a1-463c-9225-ff6438bbb704 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:31.511191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768080af-30a1-463c-9225-ff6438bbb704 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:31.511509 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768080af-30a1-463c-9225-ff6438bbb704 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:31.522578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768080af-30a1-463c-9225-ff6438bbb704 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:17:31.530365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-768080af-30a1-463c-9225-ff6438bbb704 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:17:31.530365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:31.530365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:31.530365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:31.530365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:31.530365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:31.530365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:31.530365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:31.530365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:31.530365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:31.530365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:31.530365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:31.530365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:17:31.530365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:17:31.530365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:17:31.530365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:17:31.531590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:17:31.531590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:17:31.531590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:17:31.531590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:17:31.531590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:31.531590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:31.531590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:31.531590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:31.531590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:31.531590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:31.531590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:31.531590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:31.531590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:31.531590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:31.531590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:31.531590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:31.532652 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:31.532652 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:31.532652 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:31.532652 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:31.532652 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:17:31.532652 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:17:31.532652 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:17:31.532652 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:17:31.532652 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:17:31.532652 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:17:31.532652 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:17:31.532652 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:17:31.532652 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 601/1208] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jun 30 10:17:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:31.538291 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd2b1b-de22-40b9-9886-c6f0cdb89c93 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:31.577517 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:31.637923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd2b1b-de22-40b9-9886-c6f0cdb89c93 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:31.674363 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea80add9-093d-4b18-add5-d91e8a6dde2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:31.675061 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80add9-093d-4b18-add5-d91e8a6dde2c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:31.675373 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80add9-093d-4b18-add5-d91e8a6dde2c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:31.675654 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea80add9-093d-4b18-add5-d91e8a6dde2c None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:31.676429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea80add9-093d-4b18-add5-d91e8a6dde2c None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:31.677380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea80add9-093d-4b18-add5-d91e8a6dde2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:31.678758 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea80add9-093d-4b18-add5-d91e8a6dde2c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:31.686602 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 608/1209] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jun 30 10:17:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:31.692442 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768080af-30a1-463c-9225-ff6438bbb704 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:31.742347 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:31.791344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768080af-30a1-463c-9225-ff6438bbb704 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:31.826056 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d5fd9-e0de-4d07-9def-5348da6cb6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:31.829624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d5fd9-e0de-4d07-9def-5348da6cb6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:31.829624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d5fd9-e0de-4d07-9def-5348da6cb6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:31.829624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936d5fd9-e0de-4d07-9def-5348da6cb6c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:31.837898 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d5fd9-e0de-4d07-9def-5348da6cb6c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:31.839352 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936d5fd9-e0de-4d07-9def-5348da6cb6c3 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:31.857575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-936d5fd9-e0de-4d07-9def-5348da6cb6c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:17:31.857575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:31.857575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:31.857575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:31.857575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:31.857575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:31.857575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:31.857575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:31.857575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:31.857575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:31.857575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:31.857575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:31.857575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:17:31.857575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:17:31.857575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:17:31.858666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:17:31.858666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:31.858666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:31.858666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:31.858666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:31.858666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:31.858666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:31.858666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:31.858666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:31.858666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:31.858666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:31.858666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:31.858666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:31.858666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:31.858666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:31.858666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:31.858666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:31.859628 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:17:31.859628 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:17:31.859628 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:17:31.859628 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:17:31.859628 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 602/1210] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:17:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:31.864499 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea80add9-093d-4b18-add5-d91e8a6dde2c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:31.961896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:31.964996 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea80add9-093d-4b18-add5-d91e8a6dde2c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:31.999454 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986c1ce-077e-46f3-9751-d72234d34a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:32.000110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986c1ce-077e-46f3-9751-d72234d34a8c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:32.000394 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986c1ce-077e-46f3-9751-d72234d34a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:32.000668 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4986c1ce-077e-46f3-9751-d72234d34a8c None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:32.001481 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4986c1ce-077e-46f3-9751-d72234d34a8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:32.001781 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4986c1ce-077e-46f3-9751-d72234d34a8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:32.003161 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4986c1ce-077e-46f3-9751-d72234d34a8c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:32.014375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 609/1211] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:32.020149 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d5fd9-e0de-4d07-9def-5348da6cb6c3 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:32.119375 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936d5fd9-e0de-4d07-9def-5348da6cb6c3 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:32.120775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:32.156499 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2dfce-0d76-4001-adfa-64a57197515f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:32.157381 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2dfce-0d76-4001-adfa-64a57197515f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:32.157659 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2dfce-0d76-4001-adfa-64a57197515f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:32.157933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b2dfce-0d76-4001-adfa-64a57197515f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:32.165653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b2dfce-0d76-4001-adfa-64a57197515f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:32.167134 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b2dfce-0d76-4001-adfa-64a57197515f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:32.181714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-57b2dfce-0d76-4001-adfa-64a57197515f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 10:17:32.181714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:32.181714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:32.181714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:32.181714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:32.181714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:32.181714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:32.181714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:32.181714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:32.181714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:32.181714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:32.181714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:32.181714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:17:32.181714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:17:32.181714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:17:32.181714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:17:32.182941 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:32.182941 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:32.182941 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:17:32.182941 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:17:32.182941 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:17:32.182941 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:17:32.182941 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:32.182941 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:32.182941 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:32.182941 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:32.182941 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:32.182941 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:32.182941 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:32.182941 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:32.182941 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:32.182941 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:32.182941 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:32.182941 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:32.183946 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:32.183946 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:32.183946 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:17:32.183946 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:17:32.183946 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:17:32.183946 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:17:32.183946 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:17:32.183946 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:17:32.183946 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 10:17:32.183946 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:17:32.183946 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 603/1212] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:17:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:32.188744 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986c1ce-077e-46f3-9751-d72234d34a8c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:32.306981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4986c1ce-077e-46f3-9751-d72234d34a8c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:32.340975 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb40d18-6bc3-4067-811f-e9bf19f7860c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:32.341644 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb40d18-6bc3-4067-811f-e9bf19f7860c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:32.341931 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb40d18-6bc3-4067-811f-e9bf19f7860c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:32.342212 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb40d18-6bc3-4067-811f-e9bf19f7860c None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:32.343047 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb40d18-6bc3-4067-811f-e9bf19f7860c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:32.343399 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb40d18-6bc3-4067-811f-e9bf19f7860c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:32.344849 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb40d18-6bc3-4067-811f-e9bf19f7860c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:32.362505 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 610/1213] 10.222.0.47 () {56 vars in 1143 bytes} [Thu Jun 30 10:17:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:32.368444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2dfce-0d76-4001-adfa-64a57197515f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:32.468722 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b2dfce-0d76-4001-adfa-64a57197515f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:32.503243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152207a-6a4d-4ac4-8cd0-75b214d75f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:32.503949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152207a-6a4d-4ac4-8cd0-75b214d75f0a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:32.504216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152207a-6a4d-4ac4-8cd0-75b214d75f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:32.504497 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152207a-6a4d-4ac4-8cd0-75b214d75f0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:32.535005 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:32.571755 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152207a-6a4d-4ac4-8cd0-75b214d75f0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:32.573287 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152207a-6a4d-4ac4-8cd0-75b214d75f0a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:32.671592 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 604/1214] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:17:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 10:17:32.678616 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb40d18-6bc3-4067-811f-e9bf19f7860c None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:32.698165 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:32.777425 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb40d18-6bc3-4067-811f-e9bf19f7860c None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:32.810203 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bae9f1-2ab1-4f63-9e82-e0a8ac693d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:32.810971 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bae9f1-2ab1-4f63-9e82-e0a8ac693d98 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:32.811253 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bae9f1-2ab1-4f63-9e82-e0a8ac693d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:32.811552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bae9f1-2ab1-4f63-9e82-e0a8ac693d98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:32.822192 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bae9f1-2ab1-4f63-9e82-e0a8ac693d98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:32.823694 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bae9f1-2ab1-4f63-9e82-e0a8ac693d98 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:32.840752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-26bae9f1-2ab1-4f63-9e82-e0a8ac693d98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:17:32.840752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:32.840752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:32.840752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:32.840752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:32.840752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:32.840752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:32.840752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:32.840752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:32.840752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:32.840752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:32.840752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:32.840752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:17:32.840752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:17:32.840752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:17:32.841722 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:17:32.841722 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:32.841722 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:32.841722 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:32.841722 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:32.841722 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:32.841722 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:32.841722 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:32.841722 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:32.841722 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:32.841722 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:32.841722 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:32.841722 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:32.841722 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:32.841722 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:32.841722 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:32.841722 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:32.842640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:17:32.842640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:17:32.842640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:17:32.842640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:17:32.842640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 611/1215] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:17:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:32.847684 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152207a-6a4d-4ac4-8cd0-75b214d75f0a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:32.907911 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:32.945524 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152207a-6a4d-4ac4-8cd0-75b214d75f0a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:32.978792 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912620c-6c3e-42e1-b33f-d548b36cf759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:32.979491 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912620c-6c3e-42e1-b33f-d548b36cf759 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:32.979779 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912620c-6c3e-42e1-b33f-d548b36cf759 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:32.980057 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c912620c-6c3e-42e1-b33f-d548b36cf759 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:32.980769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912620c-6c3e-42e1-b33f-d548b36cf759 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:32.981075 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912620c-6c3e-42e1-b33f-d548b36cf759 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:32.982476 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c912620c-6c3e-42e1-b33f-d548b36cf759 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:32.994032 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 605/1216] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:17:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:32.999832 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bae9f1-2ab1-4f63-9e82-e0a8ac693d98 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:33.068720 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:33.099290 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bae9f1-2ab1-4f63-9e82-e0a8ac693d98 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:33.132471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03314266-f68f-4b9d-9a39-5d0eacf60649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:33.133292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03314266-f68f-4b9d-9a39-5d0eacf60649 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:33.133579 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03314266-f68f-4b9d-9a39-5d0eacf60649 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:33.133862 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03314266-f68f-4b9d-9a39-5d0eacf60649 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:33.140524 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03314266-f68f-4b9d-9a39-5d0eacf60649 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:33.142010 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03314266-f68f-4b9d-9a39-5d0eacf60649 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:33.156354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-03314266-f68f-4b9d-9a39-5d0eacf60649 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:17:33.156354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:33.156354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:33.156354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:33.156354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:33.156354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:33.156354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:33.156354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:33.156354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:33.156354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:33.156354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:33.156354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:33.156354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:17:33.156354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:17:33.156354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:17:33.157467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:17:33.157467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:33.157467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:33.157467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:33.157467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:33.157467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:33.157467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:33.157467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:33.157467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:33.157467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:33.157467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:33.157467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:33.157467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:33.157467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:33.157467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:33.157467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:33.157467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:33.158582 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:17:33.158582 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:17:33.158582 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:17:33.158582 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:17:33.158582 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:17:33.158582 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:17:33.158582 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:17:33.158582 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:17:33.158582 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 612/1217] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:17:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:33.164008 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912620c-6c3e-42e1-b33f-d548b36cf759 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:33.263016 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c912620c-6c3e-42e1-b33f-d548b36cf759 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:33.284890 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:33.296942 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e818946-9ab7-4f7a-a390-bf95f6e2fd2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:33.297602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e818946-9ab7-4f7a-a390-bf95f6e2fd2e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:33.297918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e818946-9ab7-4f7a-a390-bf95f6e2fd2e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:33.298200 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e818946-9ab7-4f7a-a390-bf95f6e2fd2e None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:33.299013 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e818946-9ab7-4f7a-a390-bf95f6e2fd2e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:33.299356 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e818946-9ab7-4f7a-a390-bf95f6e2fd2e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:33.300797 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e818946-9ab7-4f7a-a390-bf95f6e2fd2e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:33.313079 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 606/1218] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:17:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:33.320424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03314266-f68f-4b9d-9a39-5d0eacf60649 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:33.417447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03314266-f68f-4b9d-9a39-5d0eacf60649 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:33.444435 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:33.449927 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1acf70-1690-4037-b619-d29c225cf654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:33.450566 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1acf70-1690-4037-b619-d29c225cf654 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:33.450847 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1acf70-1690-4037-b619-d29c225cf654 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:33.451121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1acf70-1690-4037-b619-d29c225cf654 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:33.451973 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1acf70-1690-4037-b619-d29c225cf654 None admin] RBAC: Adding query filter params (role.id=99d8cd9cd87e472290bf0de2710b5857, scope.project.id=50902e0539434c82b505dc5009aeaf85, user.id=46c99f43a9344d7d92632990e45bb9cf) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:33.452288 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1acf70-1690-4037-b619-d29c225cf654 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:33.453880 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1acf70-1690-4037-b619-d29c225cf654 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:33.460108 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 613/1219] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:17:33 2022] GET /identity/v3/role_assignments?user.id=46c99f43a9344d7d92632990e45bb9cf&scope.project.id=50902e0539434c82b505dc5009aeaf85&role.id=99d8cd9cd87e472290bf0de2710b5857 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:35.453510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3647518-d81b-4678-b4d7-0f4e800ef428 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:35.454100 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3647518-d81b-4678-b4d7-0f4e800ef428 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:35.454100 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3647518-d81b-4678-b4d7-0f4e800ef428 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:35.455452 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 607/1220] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:17:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:17:35.461506 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a87f57-a508-4d4e-978b-ca9402fcfb1d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:35.461919 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a87f57-a508-4d4e-978b-ca9402fcfb1d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:35.462104 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a87f57-a508-4d4e-978b-ca9402fcfb1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:35.479725 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-46a87f57-a508-4d4e-978b-ca9402fcfb1d None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:17:35.480966 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-46a87f57-a508-4d4e-978b-ca9402fcfb1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:17:35.548189 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 614/1221] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:17:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:17:35.571661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3647518-d81b-4678-b4d7-0f4e800ef428 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:35.672535 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3647518-d81b-4678-b4d7-0f4e800ef428 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:35.705020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0bc35-76af-41be-8b56-2c65ccc9e3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:35.705737 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0bc35-76af-41be-8b56-2c65ccc9e3e1 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:35.706016 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0bc35-76af-41be-8b56-2c65ccc9e3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:35.706287 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c0bc35-76af-41be-8b56-2c65ccc9e3e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:35.720375 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c0bc35-76af-41be-8b56-2c65ccc9e3e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:17:35.732539 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-b2c0bc35-76af-41be-8b56-2c65ccc9e3e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:17:35.732539 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:35.732539 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:35.732539 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:35.732539 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:35.732539 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:35.732539 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:35.732539 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:35.732539 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:35.732539 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:35.732539 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:35.732539 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:35.732539 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:17:35.732539 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:17:35.732539 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:17:35.732539 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:17:35.733762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:17:35.733762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:17:35.733762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:17:35.733762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:17:35.733762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:35.733762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:35.733762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:35.733762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:35.733762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:35.733762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:35.733762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:35.733762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:35.733762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:35.733762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:35.733762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:35.733762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:35.734782 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:35.734782 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:35.734782 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:35.734782 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:35.734782 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:17:35.734782 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:17:35.734782 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:17:35.734782 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:17:35.734782 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:17:35.734782 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:17:35.734782 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:17:35.734782 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:17:35.734782 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 608/1222] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jun 30 10:17:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:35.740419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a87f57-a508-4d4e-978b-ca9402fcfb1d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:35.837504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a87f57-a508-4d4e-978b-ca9402fcfb1d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:35.860964 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:35.871099 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b258108-e53f-4a16-b7aa-ad790d817690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:35.871787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b258108-e53f-4a16-b7aa-ad790d817690 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:35.872068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b258108-e53f-4a16-b7aa-ad790d817690 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:35.872340 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b258108-e53f-4a16-b7aa-ad790d817690 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:35.873086 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b258108-e53f-4a16-b7aa-ad790d817690 None admin] RBAC: Adding query filter params (name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:35.873391 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b258108-e53f-4a16-b7aa-ad790d817690 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:35.874773 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b258108-e53f-4a16-b7aa-ad790d817690 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:35.882384 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 615/1223] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jun 30 10:17:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:35.888402 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0bc35-76af-41be-8b56-2c65ccc9e3e1 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:35.933248 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:35.989051 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c0bc35-76af-41be-8b56-2c65ccc9e3e1 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:36.022659 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09d8f7-9215-44cd-8db8-4fc098bc0dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:36.023479 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09d8f7-9215-44cd-8db8-4fc098bc0dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:36.023767 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09d8f7-9215-44cd-8db8-4fc098bc0dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:36.024054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e09d8f7-9215-44cd-8db8-4fc098bc0dc2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:36.034211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09d8f7-9215-44cd-8db8-4fc098bc0dc2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:36.035679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e09d8f7-9215-44cd-8db8-4fc098bc0dc2 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:36.052292 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-3e09d8f7-9215-44cd-8db8-4fc098bc0dc2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:17:36.052292 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:36.052292 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:36.052292 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:36.052292 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:36.052292 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:36.052292 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:36.052292 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:36.052292 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:36.052292 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:36.052292 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:36.052292 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:36.052292 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:17:36.052292 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:17:36.052292 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:17:36.053152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:17:36.053152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:36.053152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:36.053152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:36.053152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:36.053152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:36.053152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:36.053152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:36.053152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:36.053152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:36.053152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:36.053152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:36.053152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:36.053152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:36.053152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:36.053152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:36.053152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:36.054068 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:17:36.054068 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:17:36.054068 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:17:36.054068 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:17:36.054068 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 609/1224] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:17:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:36.059828 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b258108-e53f-4a16-b7aa-ad790d817690 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:36.158119 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b258108-e53f-4a16-b7aa-ad790d817690 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:36.192521 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a2480-30df-4764-8a3e-88273566ee51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:36.193199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a2480-30df-4764-8a3e-88273566ee51 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:36.193483 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a2480-30df-4764-8a3e-88273566ee51 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:36.193760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719a2480-30df-4764-8a3e-88273566ee51 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:36.194480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a2480-30df-4764-8a3e-88273566ee51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:36.194787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a2480-30df-4764-8a3e-88273566ee51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:36.196244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719a2480-30df-4764-8a3e-88273566ee51 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:36.208444 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 616/1225] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:36.214169 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09d8f7-9215-44cd-8db8-4fc098bc0dc2 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:36.308230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:36.314652 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e09d8f7-9215-44cd-8db8-4fc098bc0dc2 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:36.347821 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd92a2e-72ce-4b40-81db-55333191237f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:36.348684 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd92a2e-72ce-4b40-81db-55333191237f None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:36.349052 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd92a2e-72ce-4b40-81db-55333191237f None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:36.349337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd92a2e-72ce-4b40-81db-55333191237f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:36.357871 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd92a2e-72ce-4b40-81db-55333191237f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:36.359350 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd92a2e-72ce-4b40-81db-55333191237f None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:36.376439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-1cd92a2e-72ce-4b40-81db-55333191237f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 10:17:36.376439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:36.376439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:36.376439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:36.376439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:36.376439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:36.376439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:36.376439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:36.376439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:36.376439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:36.376439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:36.376439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:36.376439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:17:36.376439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:17:36.376439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:17:36.376439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:17:36.380638 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:36.380638 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:36.380638 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:17:36.380638 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:17:36.380638 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:17:36.380638 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:17:36.380638 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:36.380638 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:36.380638 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:36.380638 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:36.380638 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:36.380638 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:36.380638 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:36.380638 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:36.380638 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:36.380638 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:36.380638 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:36.380638 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:36.381819 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:36.381819 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:36.381819 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:17:36.381819 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:17:36.381819 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:17:36.381819 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:17:36.381819 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:17:36.381819 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:17:36.381819 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 10:17:36.381819 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:17:36.381819 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 610/1226] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:17:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:36.384454 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a2480-30df-4764-8a3e-88273566ee51 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:36.430258 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:36.480064 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719a2480-30df-4764-8a3e-88273566ee51 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:36.512914 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae4b72-9983-405e-acbb-4f6f16070d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:36.513582 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae4b72-9983-405e-acbb-4f6f16070d8a None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:36.513861 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae4b72-9983-405e-acbb-4f6f16070d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:36.514134 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eae4b72-9983-405e-acbb-4f6f16070d8a None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:36.514965 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae4b72-9983-405e-acbb-4f6f16070d8a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:36.515288 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae4b72-9983-405e-acbb-4f6f16070d8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:36.516721 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eae4b72-9983-405e-acbb-4f6f16070d8a None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:36.533441 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 617/1227] 10.222.0.47 () {56 vars in 1143 bytes} [Thu Jun 30 10:17:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:17:36.539220 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd92a2e-72ce-4b40-81db-55333191237f None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:36.637676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd92a2e-72ce-4b40-81db-55333191237f None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:36.670962 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29df447-653e-4041-b2fd-a6ed16cfed9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:36.671777 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29df447-653e-4041-b2fd-a6ed16cfed9e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:36.672061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29df447-653e-4041-b2fd-a6ed16cfed9e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:36.672338 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29df447-653e-4041-b2fd-a6ed16cfed9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:36.697387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29df447-653e-4041-b2fd-a6ed16cfed9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:36.698819 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29df447-653e-4041-b2fd-a6ed16cfed9e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:36.730227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-d29df447-653e-4041-b2fd-a6ed16cfed9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:17:36.730227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:36.730227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:36.730227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:36.730227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:36.730227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:36.730227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:36.730227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:36.730227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:36.730227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:36.730227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:36.730227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:36.730227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:17:36.730227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:17:36.730227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:17:36.731408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:17:36.731408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:36.731408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:36.731408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:36.731408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:36.731408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:36.731408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:36.731408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:36.731408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:36.731408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:36.731408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:36.731408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:36.731408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:36.731408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:36.731408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:36.731408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:36.731408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:36.732623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:17:36.732623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:17:36.732623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:17:36.732623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:17:36.732623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 611/1228] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:17:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:36.737209 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae4b72-9983-405e-acbb-4f6f16070d8a None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:36.835319 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eae4b72-9983-405e-acbb-4f6f16070d8a None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:36.837596 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:36.869150 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781fefa0-6520-4e03-9494-c3c84c57fbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:36.869800 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fefa0-6520-4e03-9494-c3c84c57fbdf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:36.870079 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fefa0-6520-4e03-9494-c3c84c57fbdf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:36.870350 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fefa0-6520-4e03-9494-c3c84c57fbdf None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:36.871059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781fefa0-6520-4e03-9494-c3c84c57fbdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:36.871389 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781fefa0-6520-4e03-9494-c3c84c57fbdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:36.872746 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781fefa0-6520-4e03-9494-c3c84c57fbdf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:36.884564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 618/1229] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:17:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:36.893633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29df447-653e-4041-b2fd-a6ed16cfed9e None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:36.900001 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:36.989857 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29df447-653e-4041-b2fd-a6ed16cfed9e None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:37.024152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f51a9-04ef-407b-a553-b4b6d02ddc70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:37.024952 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f51a9-04ef-407b-a553-b4b6d02ddc70 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:37.025239 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f51a9-04ef-407b-a553-b4b6d02ddc70 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:37.025517 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f51a9-04ef-407b-a553-b4b6d02ddc70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:37.032115 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f51a9-04ef-407b-a553-b4b6d02ddc70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:37.033557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f51a9-04ef-407b-a553-b4b6d02ddc70 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:37.048225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-1e5f51a9-04ef-407b-a553-b4b6d02ddc70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:17:37.048225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:37.048225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:37.048225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:37.048225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:37.048225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:37.048225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:37.048225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:37.048225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:37.048225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:37.048225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:37.048225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:37.048225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:17:37.048225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:17:37.048225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:17:37.049240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:17:37.049240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:37.049240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:37.049240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:37.049240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:37.049240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:37.049240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:37.049240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:37.049240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:37.049240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:37.049240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:37.049240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:37.049240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:37.049240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:37.049240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:37.049240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:37.049240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:37.050513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:17:37.050513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:17:37.050513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:17:37.050513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:17:37.050513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:17:37.050513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:17:37.050513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:17:37.050513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:17:37.050513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 612/1230] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:17:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:37.056191 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781fefa0-6520-4e03-9494-c3c84c57fbdf None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:37.154407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781fefa0-6520-4e03-9494-c3c84c57fbdf None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:37.187042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2a1cbc-e8b4-4f2c-8664-0fc5ea613af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:37.187747 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a1cbc-e8b4-4f2c-8664-0fc5ea613af4 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:37.188036 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a1cbc-e8b4-4f2c-8664-0fc5ea613af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:37.188314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a1cbc-e8b4-4f2c-8664-0fc5ea613af4 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:37.189106 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2a1cbc-e8b4-4f2c-8664-0fc5ea613af4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:37.189419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2a1cbc-e8b4-4f2c-8664-0fc5ea613af4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:37.190868 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2a1cbc-e8b4-4f2c-8664-0fc5ea613af4 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:37.202866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 619/1231] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:17:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:37.209509 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f51a9-04ef-407b-a553-b4b6d02ddc70 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:37.276319 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:37.310155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f51a9-04ef-407b-a553-b4b6d02ddc70 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:37.343940 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9f7e7d-bcda-4686-86c4-a9aa4025de5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:37.344826 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f7e7d-bcda-4686-86c4-a9aa4025de5e None admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:37.345170 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f7e7d-bcda-4686-86c4-a9aa4025de5e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:37.345453 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9f7e7d-bcda-4686-86c4-a9aa4025de5e None admin] PATH_INFO: `/v3/projects/50902e0539434c82b505dc5009aeaf85/users/46c99f43a9344d7d92632990e45bb9cf/roles/99d8cd9cd87e472290bf0de2710b5857` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:37.359781 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9f7e7d-bcda-4686-86c4-a9aa4025de5e None admin] RBAC: Authorizing `identity:create_grant(project_id=50902e0539434c82b505dc5009aeaf85, user_id=46c99f43a9344d7d92632990e45bb9cf, role_id=99d8cd9cd87e472290bf0de2710b5857)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:37.361227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9f7e7d-bcda-4686-86c4-a9aa4025de5e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:37.524107 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 613/1232] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jun 30 10:17:37 2022] PUT /identity/v3/projects/50902e0539434c82b505dc5009aeaf85/users/46c99f43a9344d7d92632990e45bb9cf/roles/99d8cd9cd87e472290bf0de2710b5857 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:17:39.534447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f233a-4968-4f99-9cac-bd8a4c8c651a None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:39.534944 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f233a-4968-4f99-9cac-bd8a4c8c651a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:39.535051 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05f233a-4968-4f99-9cac-bd8a4c8c651a None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:39.536960 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 620/1233] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:17:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:17:39.542217 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cbe87-0dff-4457-b47d-a499cbf4e51e None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:39.542661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cbe87-0dff-4457-b47d-a499cbf4e51e None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:39.542803 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3cbe87-0dff-4457-b47d-a499cbf4e51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:39.559825 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-1b3cbe87-0dff-4457-b47d-a499cbf4e51e None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:17:39.561083 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-1b3cbe87-0dff-4457-b47d-a499cbf4e51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:17:39.629124 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 614/1234] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:17:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:17:39.653947 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f233a-4968-4f99-9cac-bd8a4c8c651a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:39.679741 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:39.755824 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05f233a-4968-4f99-9cac-bd8a4c8c651a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:39.790060 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21045951-68b4-4e62-ab91-01fec592deb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:39.790854 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21045951-68b4-4e62-ab91-01fec592deb0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:39.791138 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21045951-68b4-4e62-ab91-01fec592deb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:39.791453 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21045951-68b4-4e62-ab91-01fec592deb0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:39.801418 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21045951-68b4-4e62-ab91-01fec592deb0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:17:39.808997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-21045951-68b4-4e62-ab91-01fec592deb0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:17:39.808997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:39.808997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:39.808997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:39.808997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:39.808997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:39.808997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:39.808997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:39.808997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:39.808997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:39.808997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:39.808997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:39.808997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:17:39.808997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:17:39.808997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:17:39.808997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:17:39.810102 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:17:39.810102 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:17:39.810102 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:17:39.810102 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:17:39.810102 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:39.810102 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:39.810102 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:39.810102 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:39.810102 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:39.810102 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:39.810102 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:39.810102 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:39.810102 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:39.810102 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:39.810102 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:39.810102 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:39.811173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:39.811173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:39.811173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:39.811173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:39.811173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:17:39.811173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:17:39.811173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:17:39.811173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:17:39.811173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:17:39.811173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:17:39.811173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 30 10:17:39.811173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:17:39.811173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 621/1235] 10.222.0.47 () {56 vars in 1102 bytes} [Thu Jun 30 10:17:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:39.816346 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3cbe87-0dff-4457-b47d-a499cbf4e51e None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:39.862745 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:39.915023 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3cbe87-0dff-4457-b47d-a499cbf4e51e None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:39.947013 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523c720-17d8-43e1-990b-71c1071676e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:39.947704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523c720-17d8-43e1-990b-71c1071676e5 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:39.947991 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523c720-17d8-43e1-990b-71c1071676e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:39.948271 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7523c720-17d8-43e1-990b-71c1071676e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:39.949017 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523c720-17d8-43e1-990b-71c1071676e5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:39.949327 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523c720-17d8-43e1-990b-71c1071676e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:39.950725 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7523c720-17d8-43e1-990b-71c1071676e5 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:39.959353 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 615/1236] 10.222.0.47 () {56 vars in 1103 bytes} [Thu Jun 30 10:17:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:17:39.965310 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21045951-68b4-4e62-ab91-01fec592deb0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:40.050271 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:40.065830 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21045951-68b4-4e62-ab91-01fec592deb0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:40.098747 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf153039-2552-442f-a4ce-43aedbef48de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:40.099548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf153039-2552-442f-a4ce-43aedbef48de None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:40.099872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf153039-2552-442f-a4ce-43aedbef48de None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:40.100147 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf153039-2552-442f-a4ce-43aedbef48de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:40.110163 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf153039-2552-442f-a4ce-43aedbef48de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:40.111815 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf153039-2552-442f-a4ce-43aedbef48de None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:40.154653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-bf153039-2552-442f-a4ce-43aedbef48de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:17:40.154653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:40.154653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:40.154653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:40.154653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:40.154653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:40.154653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:40.154653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:40.154653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:40.154653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:40.154653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:40.154653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:40.154653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:17:40.154653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:17:40.154653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:17:40.155778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:17:40.155778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:40.155778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:40.155778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:40.155778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:40.155778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:40.155778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:40.155778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:40.155778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:40.155778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:40.155778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:40.155778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:40.155778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:40.155778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:40.155778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:40.155778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:40.155778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:40.156883 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:17:40.156883 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:17:40.156883 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:17:40.156883 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:17:40.156883 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 622/1237] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:17:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:40.157485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523c720-17d8-43e1-990b-71c1071676e5 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:40.238357 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7523c720-17d8-43e1-990b-71c1071676e5 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:40.248229 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:40.273328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f1e00-f972-4b2d-8295-7ce795d2398b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:40.273986 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f1e00-f972-4b2d-8295-7ce795d2398b None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:40.274272 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f1e00-f972-4b2d-8295-7ce795d2398b None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:40.274549 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f1e00-f972-4b2d-8295-7ce795d2398b None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:40.275276 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f1e00-f972-4b2d-8295-7ce795d2398b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:40.275599 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f1e00-f972-4b2d-8295-7ce795d2398b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:40.276959 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f1e00-f972-4b2d-8295-7ce795d2398b None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:40.288075 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 616/1238] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:17:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:40.293775 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf153039-2552-442f-a4ce-43aedbef48de None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:40.391326 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf153039-2552-442f-a4ce-43aedbef48de None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:40.435281 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf3c4e-95d5-46a9-b723-d80fdc68606d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:40.436112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf3c4e-95d5-46a9-b723-d80fdc68606d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:40.436397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf3c4e-95d5-46a9-b723-d80fdc68606d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:40.436676 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cf3c4e-95d5-46a9-b723-d80fdc68606d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:40.439530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:40.444938 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cf3c4e-95d5-46a9-b723-d80fdc68606d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:40.447384 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cf3c4e-95d5-46a9-b723-d80fdc68606d None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:40.460820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-12cf3c4e-95d5-46a9-b723-d80fdc68606d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 10:17:40.460820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:40.460820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:40.460820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:40.460820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:40.460820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:40.460820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:40.460820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:40.460820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:40.460820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:40.460820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:40.460820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:40.460820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:17:40.460820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:17:40.460820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:17:40.460820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:17:40.461999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:40.461999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:40.461999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:17:40.461999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:17:40.461999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:17:40.461999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:17:40.461999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:40.461999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:40.461999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:40.461999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:40.461999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:40.461999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:40.461999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:40.461999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:40.461999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:40.461999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:40.461999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:40.461999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:40.463368 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:40.463368 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:40.463368 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:17:40.463368 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:17:40.463368 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:17:40.463368 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:17:40.463368 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:17:40.463368 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:17:40.463368 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 10:17:40.463368 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:17:40.463368 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 623/1239] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:17:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:40.467924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f1e00-f972-4b2d-8295-7ce795d2398b None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:40.577284 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f1e00-f972-4b2d-8295-7ce795d2398b None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:40.611650 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630ea92d-5621-48d0-9f8f-2fc9a0f5daf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:40.612331 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ea92d-5621-48d0-9f8f-2fc9a0f5daf9 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:40.612691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ea92d-5621-48d0-9f8f-2fc9a0f5daf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:40.612893 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630ea92d-5621-48d0-9f8f-2fc9a0f5daf9 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:40.613785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630ea92d-5621-48d0-9f8f-2fc9a0f5daf9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:40.614307 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630ea92d-5621-48d0-9f8f-2fc9a0f5daf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:40.617036 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630ea92d-5621-48d0-9f8f-2fc9a0f5daf9 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:40.646613 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 617/1240] 10.222.0.47 () {56 vars in 1143 bytes} [Thu Jun 30 10:17:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:17:40.652665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf3c4e-95d5-46a9-b723-d80fdc68606d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:40.769664 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cf3c4e-95d5-46a9-b723-d80fdc68606d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:40.809173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3191acb-c316-47e9-8456-e41c3de73d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:40.809832 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3191acb-c316-47e9-8456-e41c3de73d51 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:40.810116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3191acb-c316-47e9-8456-e41c3de73d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:40.810395 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3191acb-c316-47e9-8456-e41c3de73d51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:40.880307 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:40.886872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3191acb-c316-47e9-8456-e41c3de73d51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:40.888923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3191acb-c316-47e9-8456-e41c3de73d51 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:41.002099 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 624/1241] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:17:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 10:17:41.008893 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630ea92d-5621-48d0-9f8f-2fc9a0f5daf9 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:41.038687 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:41.117961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630ea92d-5621-48d0-9f8f-2fc9a0f5daf9 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:41.157735 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341b367-ea8b-4edc-a64b-d8b57e359746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:41.158532 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341b367-ea8b-4edc-a64b-d8b57e359746 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:41.158816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341b367-ea8b-4edc-a64b-d8b57e359746 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:41.159137 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341b367-ea8b-4edc-a64b-d8b57e359746 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:41.169790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7341b367-ea8b-4edc-a64b-d8b57e359746 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:41.171374 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7341b367-ea8b-4edc-a64b-d8b57e359746 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:41.190639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-7341b367-ea8b-4edc-a64b-d8b57e359746 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:17:41.190639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:41.190639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:41.190639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:41.190639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:41.190639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:41.190639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:41.190639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:41.190639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:41.190639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:41.190639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:41.190639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:41.190639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:17:41.190639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:17:41.190639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:17:41.191793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:17:41.191793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:41.191793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:41.191793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:41.191793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:41.191793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:41.191793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:41.191793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:41.191793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:41.191793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:41.191793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:41.191793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:41.191793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:41.191793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:41.191793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:41.191793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:41.191793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:41.192916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:17:41.192916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:17:41.192916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:17:41.192916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:17:41.192916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 618/1242] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:17:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:41.199102 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3191acb-c316-47e9-8456-e41c3de73d51 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:41.307177 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:41.312558 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3191acb-c316-47e9-8456-e41c3de73d51 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:41.347904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453df70-7252-4618-b162-ccf61ee0e73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:41.349159 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453df70-7252-4618-b162-ccf61ee0e73b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:41.349440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453df70-7252-4618-b162-ccf61ee0e73b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:41.349718 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f453df70-7252-4618-b162-ccf61ee0e73b None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:41.350478 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453df70-7252-4618-b162-ccf61ee0e73b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:41.350758 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453df70-7252-4618-b162-ccf61ee0e73b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:41.352192 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f453df70-7252-4618-b162-ccf61ee0e73b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:41.363382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 625/1243] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:41.369727 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341b367-ea8b-4edc-a64b-d8b57e359746 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:41.447770 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:41.468283 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341b367-ea8b-4edc-a64b-d8b57e359746 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:41.502544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4f389-883c-4518-a81b-cf8dcad32c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:41.503513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4f389-883c-4518-a81b-cf8dcad32c65 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:41.503660 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4f389-883c-4518-a81b-cf8dcad32c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:41.503936 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4f389-883c-4518-a81b-cf8dcad32c65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:41.511018 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb4f389-883c-4518-a81b-cf8dcad32c65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:41.512543 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb4f389-883c-4518-a81b-cf8dcad32c65 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:41.526503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-2cb4f389-883c-4518-a81b-cf8dcad32c65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:17:41.526503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:41.526503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:41.526503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:41.526503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:41.526503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:41.526503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:41.526503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:41.526503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:41.526503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:41.526503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:41.526503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:41.526503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:17:41.526503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:17:41.526503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:17:41.527602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:17:41.527602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:41.527602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:41.527602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:41.527602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:41.527602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:41.527602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:41.527602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:41.527602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:41.527602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:41.527602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:41.527602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:41.527602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:41.527602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:41.527602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:41.527602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:41.527602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:41.528854 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:17:41.528854 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:17:41.528854 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:17:41.528854 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:17:41.528854 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:17:41.528854 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:17:41.528854 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:17:41.528854 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:17:41.528854 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 619/1244] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:17:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:41.533562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453df70-7252-4618-b162-ccf61ee0e73b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:41.633214 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f453df70-7252-4618-b162-ccf61ee0e73b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:41.666902 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9c341-b4a1-442a-9fb9-b4aac339017c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:41.667589 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9c341-b4a1-442a-9fb9-b4aac339017c None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:41.667871 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9c341-b4a1-442a-9fb9-b4aac339017c None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:41.668147 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9c341-b4a1-442a-9fb9-b4aac339017c None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:41.668930 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9c341-b4a1-442a-9fb9-b4aac339017c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:41.669290 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9c341-b4a1-442a-9fb9-b4aac339017c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:41.670733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9c341-b4a1-442a-9fb9-b4aac339017c None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:41.682468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 626/1245] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:17:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:41.688538 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4f389-883c-4518-a81b-cf8dcad32c65 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:41.789986 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4f389-883c-4518-a81b-cf8dcad32c65 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:41.823904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01747e4e-5a85-494a-9a1a-32b5a8e83f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:41.824603 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01747e4e-5a85-494a-9a1a-32b5a8e83f2d None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:41.824891 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01747e4e-5a85-494a-9a1a-32b5a8e83f2d None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:41.825162 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01747e4e-5a85-494a-9a1a-32b5a8e83f2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:41.825997 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01747e4e-5a85-494a-9a1a-32b5a8e83f2d None admin] RBAC: Adding query filter params (role.id=99d8cd9cd87e472290bf0de2710b5857, scope.project.id=50902e0539434c82b505dc5009aeaf85, user.id=46c99f43a9344d7d92632990e45bb9cf) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:41.826305 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01747e4e-5a85-494a-9a1a-32b5a8e83f2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:41.827767 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01747e4e-5a85-494a-9a1a-32b5a8e83f2d None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:41.834169 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 620/1246] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:17:41 2022] GET /identity/v3/role_assignments?user.id=46c99f43a9344d7d92632990e45bb9cf&scope.project.id=50902e0539434c82b505dc5009aeaf85&role.id=99d8cd9cd87e472290bf0de2710b5857 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:43.940678 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e43789-bf1d-4d90-8963-846e07c62a87 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:43.941206 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e43789-bf1d-4d90-8963-846e07c62a87 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:43.941299 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e43789-bf1d-4d90-8963-846e07c62a87 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:43.943363 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 627/1247] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:17:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:17:43.948529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e15f3-69ef-47a0-aecb-dc074e7c5db3 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:43.948931 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e15f3-69ef-47a0-aecb-dc074e7c5db3 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:43.949108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e15f3-69ef-47a0-aecb-dc074e7c5db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:43.954094 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:43.966516 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-340e15f3-69ef-47a0-aecb-dc074e7c5db3 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:17:43.967734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-340e15f3-69ef-47a0-aecb-dc074e7c5db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:17:44.033439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 621/1248] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:17:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:17:44.058281 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e43789-bf1d-4d90-8963-846e07c62a87 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:44.076921 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:44.157637 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e43789-bf1d-4d90-8963-846e07c62a87 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:44.192793 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e8ea7-9aa0-43a4-811d-3e50f96e55cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:44.193435 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e8ea7-9aa0-43a4-811d-3e50f96e55cb None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:44.193716 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e8ea7-9aa0-43a4-811d-3e50f96e55cb None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:44.193989 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e8ea7-9aa0-43a4-811d-3e50f96e55cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:44.203821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e8ea7-9aa0-43a4-811d-3e50f96e55cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:17:44.211375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-0c2e8ea7-9aa0-43a4-811d-3e50f96e55cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:17:44.211375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:44.211375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:44.211375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:44.211375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:44.211375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:44.211375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:44.211375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:44.211375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:44.211375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:44.211375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:44.211375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:44.211375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:17:44.211375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:17:44.211375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:17:44.211375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:17:44.212497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:17:44.212497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:17:44.212497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:17:44.212497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:17:44.212497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:44.212497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:44.212497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:44.212497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:44.212497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:44.212497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:44.212497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:44.212497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:44.212497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:44.212497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:44.212497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:44.212497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:44.213712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:44.213712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:44.213712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:44.213712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:44.213712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:17:44.213712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:17:44.213712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:17:44.213712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:17:44.213712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:17:44.213712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:17:44.213712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:17:44.213712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:17:44.213712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 628/1249] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:17:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:44.218540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e15f3-69ef-47a0-aecb-dc074e7c5db3 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:44.316780 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e15f3-69ef-47a0-aecb-dc074e7c5db3 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:44.344925 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:44.350695 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431281a3-c5e3-4c55-a341-82a13047b256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:44.351401 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431281a3-c5e3-4c55-a341-82a13047b256 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:44.351689 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431281a3-c5e3-4c55-a341-82a13047b256 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:44.351970 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431281a3-c5e3-4c55-a341-82a13047b256 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:44.352722 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431281a3-c5e3-4c55-a341-82a13047b256 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:44.353034 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431281a3-c5e3-4c55-a341-82a13047b256 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:44.354417 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431281a3-c5e3-4c55-a341-82a13047b256 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:44.362525 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 622/1250] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:17:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:44.368845 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e8ea7-9aa0-43a4-811d-3e50f96e55cb None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:44.444423 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:44.465909 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e8ea7-9aa0-43a4-811d-3e50f96e55cb None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:44.499339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30936d41-c365-4111-a45e-f0061ced9550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:44.500181 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30936d41-c365-4111-a45e-f0061ced9550 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:44.500466 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30936d41-c365-4111-a45e-f0061ced9550 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:44.500743 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30936d41-c365-4111-a45e-f0061ced9550 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:44.511902 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30936d41-c365-4111-a45e-f0061ced9550 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:44.513401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30936d41-c365-4111-a45e-f0061ced9550 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:44.530458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-30936d41-c365-4111-a45e-f0061ced9550 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:17:44.530458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:44.530458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:44.530458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:44.530458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:44.530458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:44.530458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:44.530458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:44.530458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:44.530458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:44.530458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:44.530458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:44.530458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:17:44.530458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:17:44.530458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:17:44.531573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:17:44.531573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:44.531573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:44.531573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:44.531573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:44.531573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:44.531573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:44.531573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:44.531573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:44.531573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:44.531573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:44.531573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:44.531573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:44.531573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:44.531573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:44.531573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:44.531573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:44.532494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:17:44.532494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:17:44.532494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:17:44.532494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:17:44.532494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 629/1251] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:17:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:44.538071 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431281a3-c5e3-4c55-a341-82a13047b256 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:44.638136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431281a3-c5e3-4c55-a341-82a13047b256 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:44.672155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698c7d3-f350-4c2c-8d99-0adc7c8ead83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:44.672819 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698c7d3-f350-4c2c-8d99-0adc7c8ead83 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:44.673105 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698c7d3-f350-4c2c-8d99-0adc7c8ead83 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:44.673381 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698c7d3-f350-4c2c-8d99-0adc7c8ead83 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:44.674101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698c7d3-f350-4c2c-8d99-0adc7c8ead83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:44.674410 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698c7d3-f350-4c2c-8d99-0adc7c8ead83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:44.675882 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698c7d3-f350-4c2c-8d99-0adc7c8ead83 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:44.690824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 623/1252] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:44.696892 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30936d41-c365-4111-a45e-f0061ced9550 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:44.800820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30936d41-c365-4111-a45e-f0061ced9550 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:44.826198 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:44.834161 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d597b-0d33-4250-8f4c-409c826a5202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:44.834958 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d597b-0d33-4250-8f4c-409c826a5202 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:44.835247 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d597b-0d33-4250-8f4c-409c826a5202 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:44.835556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418d597b-0d33-4250-8f4c-409c826a5202 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:44.842888 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d597b-0d33-4250-8f4c-409c826a5202 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:44.844381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418d597b-0d33-4250-8f4c-409c826a5202 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:44.858900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-418d597b-0d33-4250-8f4c-409c826a5202 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 10:17:44.858900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:44.858900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:44.858900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:44.858900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:44.858900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:44.858900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:44.858900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:44.858900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:44.858900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:44.858900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:44.858900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:44.858900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:17:44.858900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:17:44.858900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:17:44.858900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:17:44.860065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:44.860065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:44.860065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:17:44.860065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:17:44.860065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:17:44.860065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:17:44.860065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:44.860065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:44.860065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:44.860065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:44.860065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:44.860065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:44.860065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:44.860065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:44.860065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:44.860065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:44.860065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:44.860065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:44.861246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:44.861246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:44.861246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:17:44.861246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:17:44.861246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:17:44.861246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:17:44.861246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:17:44.861246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:17:44.861246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 10:17:44.861246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:17:44.861246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 630/1253] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:17:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:44.865999 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698c7d3-f350-4c2c-8d99-0adc7c8ead83 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:44.921150 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:44.964860 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698c7d3-f350-4c2c-8d99-0adc7c8ead83 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:44.997651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817dae01-58e1-4638-bb91-d27a29b1e598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:44.998320 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817dae01-58e1-4638-bb91-d27a29b1e598 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:44.998609 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817dae01-58e1-4638-bb91-d27a29b1e598 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:44.998887 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817dae01-58e1-4638-bb91-d27a29b1e598 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:44.999785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817dae01-58e1-4638-bb91-d27a29b1e598 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:45.000095 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817dae01-58e1-4638-bb91-d27a29b1e598 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:45.001532 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817dae01-58e1-4638-bb91-d27a29b1e598 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:45.018642 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 624/1254] 10.222.0.47 () {56 vars in 1143 bytes} [Thu Jun 30 10:17:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:45.025314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d597b-0d33-4250-8f4c-409c826a5202 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:45.122591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418d597b-0d33-4250-8f4c-409c826a5202 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:45.156069 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd5cf1-c437-4acb-a2f8-3ffcb8311a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:45.156730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd5cf1-c437-4acb-a2f8-3ffcb8311a7b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:45.156993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd5cf1-c437-4acb-a2f8-3ffcb8311a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:45.157271 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbd5cf1-c437-4acb-a2f8-3ffcb8311a7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:45.211646 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:45.223509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd5cf1-c437-4acb-a2f8-3ffcb8311a7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:45.224909 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbd5cf1-c437-4acb-a2f8-3ffcb8311a7b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:45.320472 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 631/1255] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:17:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 10:17:45.326320 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817dae01-58e1-4638-bb91-d27a29b1e598 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:45.422997 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817dae01-58e1-4638-bb91-d27a29b1e598 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:45.436263 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:45.456866 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd766cc-3c39-4fc7-b27b-f949399578df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:45.457938 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd766cc-3c39-4fc7-b27b-f949399578df None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:45.458423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd766cc-3c39-4fc7-b27b-f949399578df None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:45.458885 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd766cc-3c39-4fc7-b27b-f949399578df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:45.475144 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd766cc-3c39-4fc7-b27b-f949399578df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:45.475749 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd766cc-3c39-4fc7-b27b-f949399578df None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:45.494389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-fdd766cc-3c39-4fc7-b27b-f949399578df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:17:45.494389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:45.494389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:45.494389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:45.494389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:45.494389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:45.494389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:45.494389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:45.494389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:45.494389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:45.494389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:45.494389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:45.494389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:17:45.494389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:17:45.494389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:17:45.495388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:17:45.495388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:45.495388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:45.495388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:45.495388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:45.495388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:45.495388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:45.495388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:45.495388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:45.495388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:45.495388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:45.495388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:45.495388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:45.495388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:45.495388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:45.495388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:45.495388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:45.496379 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:17:45.496379 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:17:45.496379 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:17:45.496379 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:17:45.496379 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 625/1256] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:17:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:45.501601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd5cf1-c437-4acb-a2f8-3ffcb8311a7b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:45.587955 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:45.599936 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbd5cf1-c437-4acb-a2f8-3ffcb8311a7b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:45.633992 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877325e-1a74-4437-8001-9bf7f3a5d157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:45.634648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877325e-1a74-4437-8001-9bf7f3a5d157 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:45.634935 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877325e-1a74-4437-8001-9bf7f3a5d157 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:45.635327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877325e-1a74-4437-8001-9bf7f3a5d157 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:45.636009 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877325e-1a74-4437-8001-9bf7f3a5d157 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:45.636318 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877325e-1a74-4437-8001-9bf7f3a5d157 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:45.637682 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c877325e-1a74-4437-8001-9bf7f3a5d157 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:45.648972 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 632/1257] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:17:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:45.654662 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd766cc-3c39-4fc7-b27b-f949399578df None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:45.753878 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd766cc-3c39-4fc7-b27b-f949399578df None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:45.787290 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f6832-4f36-47f9-81d0-bdb71c7a27d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:45.788082 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f6832-4f36-47f9-81d0-bdb71c7a27d0 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:45.788368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f6832-4f36-47f9-81d0-bdb71c7a27d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:45.788648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f6832-4f36-47f9-81d0-bdb71c7a27d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:45.795314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f6832-4f36-47f9-81d0-bdb71c7a27d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:45.796788 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f6832-4f36-47f9-81d0-bdb71c7a27d0 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:45.810787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-f83f6832-4f36-47f9-81d0-bdb71c7a27d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:17:45.810787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:45.810787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:45.810787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:45.810787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:45.810787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:45.810787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:45.810787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:45.810787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:45.810787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:45.810787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:45.810787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:45.810787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:17:45.810787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:17:45.810787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:17:45.811903 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:17:45.811903 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:45.811903 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:45.811903 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:45.811903 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:45.811903 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:45.811903 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:45.811903 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:45.811903 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:45.811903 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:45.811903 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:45.811903 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:45.811903 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:45.811903 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:45.811903 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:45.811903 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:45.811903 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:45.817738 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:17:45.817738 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:17:45.817738 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:17:45.817738 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:17:45.817738 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:17:45.817738 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:17:45.817738 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:17:45.817738 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:17:45.817738 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 626/1258] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:17:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:45.818367 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877325e-1a74-4437-8001-9bf7f3a5d157 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:45.915696 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877325e-1a74-4437-8001-9bf7f3a5d157 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:45.948595 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5278a-6215-46d5-b7e8-908bbb6d3fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:45.949249 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5278a-6215-46d5-b7e8-908bbb6d3fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:45.949533 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5278a-6215-46d5-b7e8-908bbb6d3fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:45.949811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5278a-6215-46d5-b7e8-908bbb6d3fd6 None admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:45.950596 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5278a-6215-46d5-b7e8-908bbb6d3fd6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:45.950910 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5278a-6215-46d5-b7e8-908bbb6d3fd6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:45.952411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5278a-6215-46d5-b7e8-908bbb6d3fd6 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:45.965027 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 633/1259] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:17:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:45.971038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f6832-4f36-47f9-81d0-bdb71c7a27d0 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:46.004274 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:46.078016 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f6832-4f36-47f9-81d0-bdb71c7a27d0 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:46.114440 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd3320b-32bf-473e-9e26-83041f8b7fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:46.115107 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3320b-32bf-473e-9e26-83041f8b7fbf None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:46.115424 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3320b-32bf-473e-9e26-83041f8b7fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:46.115706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3320b-32bf-473e-9e26-83041f8b7fbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:46.116538 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd3320b-32bf-473e-9e26-83041f8b7fbf None admin] RBAC: Adding query filter params (role.id=e63bda042e104a4a93fba15ffcfe1183, scope.project.id=50902e0539434c82b505dc5009aeaf85, user.id=46c99f43a9344d7d92632990e45bb9cf) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:46.116852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd3320b-32bf-473e-9e26-83041f8b7fbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:46.118275 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd3320b-32bf-473e-9e26-83041f8b7fbf None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:46.123761 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 627/1260] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:17:45 2022] GET /identity/v3/role_assignments?user.id=46c99f43a9344d7d92632990e45bb9cf&scope.project.id=50902e0539434c82b505dc5009aeaf85&role.id=e63bda042e104a4a93fba15ffcfe1183 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:48.158535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f011b3-ea38-4768-84e1-14443d587570 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:48.159100 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f011b3-ea38-4768-84e1-14443d587570 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:48.159100 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f011b3-ea38-4768-84e1-14443d587570 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:48.161034 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 634/1261] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:17:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:17:48.166276 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507aa25-f7d3-4cd6-b413-7d7ed293645e None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:48.166667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507aa25-f7d3-4cd6-b413-7d7ed293645e None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:48.166834 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8507aa25-f7d3-4cd6-b413-7d7ed293645e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:48.183962 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-8507aa25-f7d3-4cd6-b413-7d7ed293645e None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:17:48.185141 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-8507aa25-f7d3-4cd6-b413-7d7ed293645e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:17:48.250356 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 628/1262] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:17:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:17:48.272973 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f011b3-ea38-4768-84e1-14443d587570 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:48.283622 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:48.374097 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f011b3-ea38-4768-84e1-14443d587570 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:48.407164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09429c2f-6b22-47d2-a3bd-0ccce0ab2f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:48.407959 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09429c2f-6b22-47d2-a3bd-0ccce0ab2f9d None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:48.408239 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09429c2f-6b22-47d2-a3bd-0ccce0ab2f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:48.408516 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09429c2f-6b22-47d2-a3bd-0ccce0ab2f9d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:48.419454 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09429c2f-6b22-47d2-a3bd-0ccce0ab2f9d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:17:48.427285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-09429c2f-6b22-47d2-a3bd-0ccce0ab2f9d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:17:48.427285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:48.427285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:48.427285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:48.427285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:48.427285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:48.427285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:48.427285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:48.427285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:48.427285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:48.427285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:48.427285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:48.427285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:17:48.427285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:17:48.427285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:17:48.427285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:17:48.428391 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:17:48.428391 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:17:48.428391 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:17:48.428391 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:17:48.428391 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:48.428391 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:48.428391 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:48.428391 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:48.428391 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:48.428391 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:48.428391 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:48.428391 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:48.428391 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:48.428391 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:48.428391 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:48.428391 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:48.429446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:48.429446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:48.429446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:48.429446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:48.429446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:17:48.429446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:17:48.429446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:17:48.429446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:17:48.429446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:17:48.429446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:17:48.429446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:17:48.429446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:17:48.429446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 635/1263] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:17:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:48.434756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507aa25-f7d3-4cd6-b413-7d7ed293645e None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:48.452637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:48.533024 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8507aa25-f7d3-4cd6-b413-7d7ed293645e None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:48.566181 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e10837-185e-4bdb-b957-979d88e72028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:48.567406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e10837-185e-4bdb-b957-979d88e72028 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:48.567913 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e10837-185e-4bdb-b957-979d88e72028 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:48.568369 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e10837-185e-4bdb-b957-979d88e72028 None admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:48.569185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e10837-185e-4bdb-b957-979d88e72028 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:48.570089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e10837-185e-4bdb-b957-979d88e72028 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:48.571492 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e10837-185e-4bdb-b957-979d88e72028 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:48.579602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 629/1264] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:17:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:48.585940 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09429c2f-6b22-47d2-a3bd-0ccce0ab2f9d None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:48.651330 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:48.683235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09429c2f-6b22-47d2-a3bd-0ccce0ab2f9d None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:48.715735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8354112-4aaf-4d36-8be5-8e431030adf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:48.716513 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8354112-4aaf-4d36-8be5-8e431030adf0 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:48.716795 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8354112-4aaf-4d36-8be5-8e431030adf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:48.717110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8354112-4aaf-4d36-8be5-8e431030adf0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:48.727594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8354112-4aaf-4d36-8be5-8e431030adf0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:48.729510 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8354112-4aaf-4d36-8be5-8e431030adf0 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:48.746590 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-d8354112-4aaf-4d36-8be5-8e431030adf0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:17:48.746590 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:48.746590 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:48.746590 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:48.746590 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:48.746590 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:48.746590 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:48.746590 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:48.746590 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:48.746590 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:48.746590 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:48.746590 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:48.746590 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:17:48.746590 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:17:48.746590 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:17:48.747685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:17:48.747685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:48.747685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:48.747685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:48.747685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:48.747685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:48.747685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:48.747685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:48.747685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:48.747685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:48.747685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:48.747685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:48.747685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:48.747685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:48.747685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:48.747685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:48.747685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:48.748633 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:17:48.748633 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:17:48.748633 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:17:48.748633 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:17:48.748633 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 636/1265] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:17:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:48.753959 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e10837-185e-4bdb-b957-979d88e72028 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:48.836283 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:48.854628 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e10837-185e-4bdb-b957-979d88e72028 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:48.888223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed37d9-c171-459d-bfa1-23093d732548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:48.888882 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed37d9-c171-459d-bfa1-23093d732548 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:48.889165 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed37d9-c171-459d-bfa1-23093d732548 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:48.889439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed37d9-c171-459d-bfa1-23093d732548 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:48.890155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed37d9-c171-459d-bfa1-23093d732548 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:48.890456 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed37d9-c171-459d-bfa1-23093d732548 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:48.891851 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed37d9-c171-459d-bfa1-23093d732548 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:48.904693 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 630/1266] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:17:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:48.910386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8354112-4aaf-4d36-8be5-8e431030adf0 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:49.010411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8354112-4aaf-4d36-8be5-8e431030adf0 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:49.030108 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:49.043513 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1785f-6926-447e-bd3e-e59257384056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:49.044338 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1785f-6926-447e-bd3e-e59257384056 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:49.044622 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1785f-6926-447e-bd3e-e59257384056 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:49.044900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1785f-6926-447e-bd3e-e59257384056 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:49.052625 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1785f-6926-447e-bd3e-e59257384056 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:49.054057 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1785f-6926-447e-bd3e-e59257384056 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:49.068597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-14b1785f-6926-447e-bd3e-e59257384056 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 10:17:49.068597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:49.068597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:49.068597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:49.068597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:49.068597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:49.068597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:49.068597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:49.068597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:49.068597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:49.068597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:49.068597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:49.068597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:17:49.068597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:17:49.068597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:17:49.068597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:17:49.078255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:49.078255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:49.078255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:17:49.078255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:17:49.078255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:17:49.078255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:17:49.078255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:49.078255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:49.078255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:49.078255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:49.078255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:49.078255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:49.078255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:49.078255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:49.078255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:49.078255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:49.078255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:49.078255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:49.079479 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:49.079479 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:49.079479 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:17:49.079479 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:17:49.079479 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:17:49.079479 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:17:49.079479 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:17:49.079479 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:17:49.079479 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 10:17:49.079479 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:17:49.079479 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 637/1267] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:17:48 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:49.080230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed37d9-c171-459d-bfa1-23093d732548 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:49.174276 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed37d9-c171-459d-bfa1-23093d732548 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:49.209052 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f1d8c-c0eb-4685-a00d-2a695f7b8a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:49.209754 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f1d8c-c0eb-4685-a00d-2a695f7b8a97 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:49.210037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f1d8c-c0eb-4685-a00d-2a695f7b8a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:49.210311 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f1d8c-c0eb-4685-a00d-2a695f7b8a97 None admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:49.211144 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f1d8c-c0eb-4685-a00d-2a695f7b8a97 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:49.211482 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f1d8c-c0eb-4685-a00d-2a695f7b8a97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:49.212928 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898f1d8c-c0eb-4685-a00d-2a695f7b8a97 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:49.221343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:49.229932 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 631/1268] 10.222.0.47 () {56 vars in 1143 bytes} [Thu Jun 30 10:17:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:49.235539 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1785f-6926-447e-bd3e-e59257384056 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:49.334618 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1785f-6926-447e-bd3e-e59257384056 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:49.367318 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2b8a5-51df-44e6-8f51-72dd06ef7455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:49.368112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2b8a5-51df-44e6-8f51-72dd06ef7455 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:49.368397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2b8a5-51df-44e6-8f51-72dd06ef7455 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:49.368662 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c2b8a5-51df-44e6-8f51-72dd06ef7455 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:49.379075 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c2b8a5-51df-44e6-8f51-72dd06ef7455 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:49.381161 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c2b8a5-51df-44e6-8f51-72dd06ef7455 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:49.398292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-f1c2b8a5-51df-44e6-8f51-72dd06ef7455 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:17:49.398292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:49.398292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:49.398292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:49.398292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:49.398292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:49.398292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:49.398292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:49.398292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:49.398292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:49.398292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:49.398292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:49.398292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:17:49.398292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:17:49.398292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:17:49.399362 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:17:49.399362 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:49.399362 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:49.399362 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:49.399362 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:49.399362 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:49.399362 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:49.399362 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:49.399362 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:49.399362 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:49.399362 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:49.399362 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:49.399362 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:49.399362 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:49.399362 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:49.399362 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:49.399362 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:49.400445 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:17:49.400445 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:17:49.400445 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:17:49.400445 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:17:49.400445 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 638/1269] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:17:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:49.405201 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f1d8c-c0eb-4685-a00d-2a695f7b8a97 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:49.506045 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f1d8c-c0eb-4685-a00d-2a695f7b8a97 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:49.541524 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41ba53-78e3-493b-8cfb-d496ec3c3065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:49.542184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41ba53-78e3-493b-8cfb-d496ec3c3065 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:49.542470 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41ba53-78e3-493b-8cfb-d496ec3c3065 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:49.542749 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41ba53-78e3-493b-8cfb-d496ec3c3065 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:49.543500 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b41ba53-78e3-493b-8cfb-d496ec3c3065 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:49.543821 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b41ba53-78e3-493b-8cfb-d496ec3c3065 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:49.545238 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b41ba53-78e3-493b-8cfb-d496ec3c3065 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:49.556824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 632/1270] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:17:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:49.562561 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2b8a5-51df-44e6-8f51-72dd06ef7455 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:49.594350 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:49.661949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c2b8a5-51df-44e6-8f51-72dd06ef7455 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:49.698467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598810c6-7431-4108-abcb-6e57e481cc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:49.699277 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598810c6-7431-4108-abcb-6e57e481cc9b None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:49.699570 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598810c6-7431-4108-abcb-6e57e481cc9b None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:49.699842 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598810c6-7431-4108-abcb-6e57e481cc9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:49.706365 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598810c6-7431-4108-abcb-6e57e481cc9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:49.707942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598810c6-7431-4108-abcb-6e57e481cc9b None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:49.721574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-598810c6-7431-4108-abcb-6e57e481cc9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:17:49.721574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:49.721574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:49.721574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:49.721574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:49.721574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:49.721574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:49.721574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:49.721574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:49.721574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:49.721574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:49.721574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:49.721574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:17:49.721574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:17:49.721574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:17:49.722623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:17:49.722623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:49.722623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:49.722623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:49.722623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:49.722623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:49.722623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:49.722623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:49.722623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:49.722623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:49.722623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:49.722623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:49.722623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:49.722623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:49.722623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:49.722623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:49.722623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:49.724626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:17:49.724626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:17:49.724626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:17:49.724626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:17:49.724626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:17:49.724626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:17:49.724626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:17:49.724626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:17:49.724626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 639/1271] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:17:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:49.729403 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41ba53-78e3-493b-8cfb-d496ec3c3065 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:49.793335 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:49.835055 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41ba53-78e3-493b-8cfb-d496ec3c3065 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:49.867333 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16719920-dfa3-431b-a10b-0b8217a4eec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:49.867938 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16719920-dfa3-431b-a10b-0b8217a4eec0 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:49.868248 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16719920-dfa3-431b-a10b-0b8217a4eec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:49.868525 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16719920-dfa3-431b-a10b-0b8217a4eec0 None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:49.869307 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16719920-dfa3-431b-a10b-0b8217a4eec0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:49.869618 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16719920-dfa3-431b-a10b-0b8217a4eec0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:49.871040 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16719920-dfa3-431b-a10b-0b8217a4eec0 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:49.882559 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 633/1272] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:17:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:49.888673 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598810c6-7431-4108-abcb-6e57e481cc9b None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:49.980728 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:49.988965 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598810c6-7431-4108-abcb-6e57e481cc9b None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:50.021357 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7ea9f-2a1f-47a4-9a32-89b185c93016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:50.022242 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7ea9f-2a1f-47a4-9a32-89b185c93016 None admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:50.022789 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7ea9f-2a1f-47a4-9a32-89b185c93016 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:50.023064 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7ea9f-2a1f-47a4-9a32-89b185c93016 None admin] PATH_INFO: `/v3/projects/50902e0539434c82b505dc5009aeaf85/users/46c99f43a9344d7d92632990e45bb9cf/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:50.025038 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb7ea9f-2a1f-47a4-9a32-89b185c93016 None admin] RBAC: Authorizing `identity:create_grant(project_id=50902e0539434c82b505dc5009aeaf85, user_id=46c99f43a9344d7d92632990e45bb9cf, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:50.026480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb7ea9f-2a1f-47a4-9a32-89b185c93016 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:50.188227 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 640/1273] 10.222.0.47 () {58 vars in 1439 bytes} [Thu Jun 30 10:17:49 2022] PUT /identity/v3/projects/50902e0539434c82b505dc5009aeaf85/users/46c99f43a9344d7d92632990e45bb9cf/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:17:52.257880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1bba8-c5b3-450c-860c-4490df33e4a0 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:52.258417 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1bba8-c5b3-450c-860c-4490df33e4a0 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:52.258417 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1bba8-c5b3-450c-860c-4490df33e4a0 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:52.259856 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 634/1274] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:17:52.264891 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35748c-7722-4932-8a1a-a948e2539cd3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:52.265323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35748c-7722-4932-8a1a-a948e2539cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:52.265443 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35748c-7722-4932-8a1a-a948e2539cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:52.282850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-de35748c-7722-4932-8a1a-a948e2539cd3 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:17:52.284044 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-de35748c-7722-4932-8a1a-a948e2539cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:17:52.351067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 641/1275] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:17:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:17:52.374558 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1bba8-c5b3-450c-860c-4490df33e4a0 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:52.480677 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1bba8-c5b3-450c-860c-4490df33e4a0 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:52.498303 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:52.516131 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968cb940-757d-4fff-b427-b1af298ec934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:52.516863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cb940-757d-4fff-b427-b1af298ec934 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:52.517149 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cb940-757d-4fff-b427-b1af298ec934 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:52.517426 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968cb940-757d-4fff-b427-b1af298ec934 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:52.528046 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968cb940-757d-4fff-b427-b1af298ec934 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 30 10:17:52.535739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-968cb940-757d-4fff-b427-b1af298ec934 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:17:52.535739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:52.535739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:52.535739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:52.535739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:52.535739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:52.535739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:52.535739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:52.535739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:52.535739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:52.535739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:52.535739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:52.535739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 30 10:17:52.535739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 30 10:17:52.535739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 30 10:17:52.535739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 30 10:17:52.536736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 30 10:17:52.536736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 30 10:17:52.536736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 30 10:17:52.536736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 30 10:17:52.536736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:52.536736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:52.536736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:52.536736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:52.536736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:52.536736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:52.536736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:52.536736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:52.536736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:52.536736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:52.536736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:52.536736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:52.537753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:52.537753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:52.537753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:52.537753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:52.537753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 30 10:17:52.537753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 30 10:17:52.537753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 30 10:17:52.537753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 30 10:17:52.537753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 30 10:17:52.537753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 30 10:17:52.537753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 30 10:17:52.537753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:17:52.537753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 635/1276] 10.222.0.47 () {56 vars in 1096 bytes} [Thu Jun 30 10:17:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:52.542921 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35748c-7722-4932-8a1a-a948e2539cd3 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:52.624465 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:52.642342 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de35748c-7722-4932-8a1a-a948e2539cd3 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:52.675343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67dd5a-be2e-42f0-84bb-7d665a910bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:52.676053 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67dd5a-be2e-42f0-84bb-7d665a910bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:52.676336 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67dd5a-be2e-42f0-84bb-7d665a910bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:52.677154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67dd5a-be2e-42f0-84bb-7d665a910bc3 None admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:52.677908 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e67dd5a-be2e-42f0-84bb-7d665a910bc3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:52.678214 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e67dd5a-be2e-42f0-84bb-7d665a910bc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:52.679603 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e67dd5a-be2e-42f0-84bb-7d665a910bc3 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:52.687405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 642/1277] 10.222.0.47 () {56 vars in 1099 bytes} [Thu Jun 30 10:17:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:52.693922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968cb940-757d-4fff-b427-b1af298ec934 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:52.799556 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968cb940-757d-4fff-b427-b1af298ec934 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:52.832427 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ee743-969b-4611-a363-f6fdc1724cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:52.833283 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ee743-969b-4611-a363-f6fdc1724cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:52.833566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ee743-969b-4611-a363-f6fdc1724cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:52.833841 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905ee743-969b-4611-a363-f6fdc1724cf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:52.844100 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ee743-969b-4611-a363-f6fdc1724cf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:52.845531 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905ee743-969b-4611-a363-f6fdc1724cf8 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:52.863423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-905ee743-969b-4611-a363-f6fdc1724cf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:17:52.863423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:52.863423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:52.863423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:52.863423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:52.863423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:52.863423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:52.863423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:52.863423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:52.863423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:52.863423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:52.863423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:52.863423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:17:52.863423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:17:52.863423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:17:52.865562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:17:52.865562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:52.865562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:52.865562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:52.865562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:52.865562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:52.865562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:52.865562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:52.865562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:52.865562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:52.865562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:52.865562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:52.865562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:52.865562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:52.865562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:52.865562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:52.865562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:52.866740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:17:52.866740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:17:52.866740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:17:52.866740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:17:52.866740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 636/1278] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:17:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:17:52.871483 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67dd5a-be2e-42f0-84bb-7d665a910bc3 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:52.970312 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67dd5a-be2e-42f0-84bb-7d665a910bc3 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:53.002837 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1ebf5-6c47-4a04-b790-d6a85092cd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:53.003526 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1ebf5-6c47-4a04-b790-d6a85092cd35 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:53.003812 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1ebf5-6c47-4a04-b790-d6a85092cd35 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:53.004090 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a1ebf5-6c47-4a04-b790-d6a85092cd35 None admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:53.004944 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a1ebf5-6c47-4a04-b790-d6a85092cd35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:53.005367 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a1ebf5-6c47-4a04-b790-d6a85092cd35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:53.007247 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a1ebf5-6c47-4a04-b790-d6a85092cd35 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:53.019344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 643/1279] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:17:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:53.025280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ee743-969b-4611-a363-f6fdc1724cf8 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:53.056639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:53.126455 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905ee743-969b-4611-a363-f6fdc1724cf8 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:53.161199 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee047bf-8ad3-4e03-b777-e653e6632102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:53.162122 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee047bf-8ad3-4e03-b777-e653e6632102 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:53.162410 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee047bf-8ad3-4e03-b777-e653e6632102 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:53.162692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee047bf-8ad3-4e03-b777-e653e6632102 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:53.169969 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee047bf-8ad3-4e03-b777-e653e6632102 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:53.171440 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee047bf-8ad3-4e03-b777-e653e6632102 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:53.185974 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-0ee047bf-8ad3-4e03-b777-e653e6632102 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 30 10:17:53.185974 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:53.185974 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:53.185974 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:53.185974 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:53.185974 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:53.185974 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:53.185974 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:53.185974 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:53.185974 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:53.185974 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:53.185974 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:53.185974 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 30 10:17:53.185974 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 30 10:17:53.185974 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 30 10:17:53.185974 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 30 10:17:53.187163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:53.187163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:53.187163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 30 10:17:53.187163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:17:53.187163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 30 10:17:53.187163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 30 10:17:53.187163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:53.187163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:53.187163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:53.187163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:53.187163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:53.187163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:53.187163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:53.187163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:53.187163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:53.187163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:53.187163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:53.187163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:53.188301 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:53.188301 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:53.188301 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 30 10:17:53.188301 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 30 10:17:53.188301 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 30 10:17:53.188301 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 30 10:17:53.188301 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 30 10:17:53.188301 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 30 10:17:53.188301 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 30 10:17:53.188301 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:17:53.188301 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 637/1280] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:17:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:17:53.192953 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1ebf5-6c47-4a04-b790-d6a85092cd35 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:53.196770 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:53.294254 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a1ebf5-6c47-4a04-b790-d6a85092cd35 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:53.327860 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a19d6-340c-4443-a76e-750cf0436b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:53.328441 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a19d6-340c-4443-a76e-750cf0436b00 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:53.328728 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a19d6-340c-4443-a76e-750cf0436b00 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:53.329017 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a19d6-340c-4443-a76e-750cf0436b00 None admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:53.329842 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a19d6-340c-4443-a76e-750cf0436b00 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:53.330153 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a19d6-340c-4443-a76e-750cf0436b00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:53.331620 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a19d6-340c-4443-a76e-750cf0436b00 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:53.348893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 644/1281] 10.222.0.47 () {56 vars in 1143 bytes} [Thu Jun 30 10:17:53 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:53.354851 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee047bf-8ad3-4e03-b777-e653e6632102 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:53.449722 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:53.459170 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee047bf-8ad3-4e03-b777-e653e6632102 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:53.494497 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ce587-97f4-4371-9d9c-527d4ca00573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:53.495184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ce587-97f4-4371-9d9c-527d4ca00573 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:53.495499 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ce587-97f4-4371-9d9c-527d4ca00573 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:53.495770 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432ce587-97f4-4371-9d9c-527d4ca00573 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:53.562021 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ce587-97f4-4371-9d9c-527d4ca00573 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:53.563503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432ce587-97f4-4371-9d9c-527d4ca00573 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:53.651834 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:53.662308 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 638/1282] 10.222.0.47 () {58 vars in 1282 bytes} [Thu Jun 30 10:17:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 30 10:17:53.668202 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a19d6-340c-4443-a76e-750cf0436b00 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:53.735142 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:53.772925 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a19d6-340c-4443-a76e-750cf0436b00 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:53.808192 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8ebf9-87d5-4c70-bfc1-6b27bc6b98db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:53.808981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8ebf9-87d5-4c70-bfc1-6b27bc6b98db None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:53.809269 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8ebf9-87d5-4c70-bfc1-6b27bc6b98db None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:53.809550 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8ebf9-87d5-4c70-bfc1-6b27bc6b98db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:53.819882 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8ebf9-87d5-4c70-bfc1-6b27bc6b98db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:53.821446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d8ebf9-87d5-4c70-bfc1-6b27bc6b98db None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:53.838830 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-22d8ebf9-87d5-4c70-bfc1-6b27bc6b98db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:17:53.838830 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:53.838830 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:53.838830 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:53.838830 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:53.838830 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:53.838830 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:53.838830 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:53.838830 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:53.838830 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:53.838830 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:53.838830 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:53.838830 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 30 10:17:53.838830 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 30 10:17:53.838830 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 30 10:17:53.839896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 30 10:17:53.839896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:53.839896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:53.839896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:53.839896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:53.839896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:53.839896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:53.839896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:53.839896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:53.839896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:53.839896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:53.839896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:53.839896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:53.839896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:53.839896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:53.839896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:53.839896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:53.840802 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 30 10:17:53.840802 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 30 10:17:53.840802 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 30 10:17:53.840802 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:17:53.840802 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 645/1283] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:17:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 30 10:17:53.845763 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ce587-97f4-4371-9d9c-527d4ca00573 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:53.946522 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432ce587-97f4-4371-9d9c-527d4ca00573 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:53.979909 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f704f1-5390-49f2-b392-5d44ac84c6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:53.980567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f704f1-5390-49f2-b392-5d44ac84c6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:53.980856 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f704f1-5390-49f2-b392-5d44ac84c6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:53.981135 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f704f1-5390-49f2-b392-5d44ac84c6b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:53.981857 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f704f1-5390-49f2-b392-5d44ac84c6b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:53.982166 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f704f1-5390-49f2-b392-5d44ac84c6b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:53.983609 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f704f1-5390-49f2-b392-5d44ac84c6b1 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:53.994849 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 639/1284] 10.222.0.47 () {56 vars in 1109 bytes} [Thu Jun 30 10:17:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:54.000512 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8ebf9-87d5-4c70-bfc1-6b27bc6b98db None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:54.102448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8ebf9-87d5-4c70-bfc1-6b27bc6b98db None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:54.115697 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:54.135415 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400befc-d6f2-4bbb-ac88-741caacdf5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:54.136199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400befc-d6f2-4bbb-ac88-741caacdf5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:54.136484 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400befc-d6f2-4bbb-ac88-741caacdf5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:54.136761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400befc-d6f2-4bbb-ac88-741caacdf5d1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:54.143184 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5400befc-d6f2-4bbb-ac88-741caacdf5d1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:54.144732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5400befc-d6f2-4bbb-ac88-741caacdf5d1 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:54.159591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application [None req-5400befc-d6f2-4bbb-ac88-741caacdf5d1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:17:54.159591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:54.159591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:54.159591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:54.159591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:54.159591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:54.159591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:54.159591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:54.159591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:54.159591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:54.159591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:54.159591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:54.159591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 30 10:17:54.159591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 30 10:17:54.159591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 30 10:17:54.160813 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 30 10:17:54.160813 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:54.160813 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:54.160813 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:54.160813 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:54.160813 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:54.160813 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:54.160813 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:54.160813 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:54.160813 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:54.160813 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:54.160813 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:54.160813 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:54.160813 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:54.160813 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:54.160813 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:54.160813 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:54.161963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 30 10:17:54.161963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 30 10:17:54.161963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 30 10:17:54.161963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 30 10:17:54.161963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 30 10:17:54.161963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 30 10:17:54.161963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 30 10:17:54.161963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: ERROR keystone.server.flask.application Jun 30 10:17:54.161963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 646/1285] 10.222.0.47 () {56 vars in 1111 bytes} [Thu Jun 30 10:17:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:54.167109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f704f1-5390-49f2-b392-5d44ac84c6b1 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:54.217055 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:54.264448 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f704f1-5390-49f2-b392-5d44ac84c6b1 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:54.297791 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcea7d97-778c-4dc1-8e93-9a7bdff1bf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:54.298448 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea7d97-778c-4dc1-8e93-9a7bdff1bf05 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:54.298732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea7d97-778c-4dc1-8e93-9a7bdff1bf05 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:54.299008 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcea7d97-778c-4dc1-8e93-9a7bdff1bf05 None admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:54.299815 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea7d97-778c-4dc1-8e93-9a7bdff1bf05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:54.300125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea7d97-778c-4dc1-8e93-9a7bdff1bf05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:54.302178 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcea7d97-778c-4dc1-8e93-9a7bdff1bf05 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:54.313716 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 640/1286] 10.222.0.47 () {56 vars in 1148 bytes} [Thu Jun 30 10:17:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:54.320508 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400befc-d6f2-4bbb-ac88-741caacdf5d1 None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:54.420128 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400befc-d6f2-4bbb-ac88-741caacdf5d1 None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:54.453659 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5aa1f-7143-4ff6-b149-08d66fcd8bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:54.454305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5aa1f-7143-4ff6-b149-08d66fcd8bbf None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:54.454591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5aa1f-7143-4ff6-b149-08d66fcd8bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:54.454867 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5aa1f-7143-4ff6-b149-08d66fcd8bbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:54.455749 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f5aa1f-7143-4ff6-b149-08d66fcd8bbf None admin] RBAC: Adding query filter params (role.id=e63bda042e104a4a93fba15ffcfe1183, scope.project.id=50902e0539434c82b505dc5009aeaf85, user.id=46c99f43a9344d7d92632990e45bb9cf) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:54.456063 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f5aa1f-7143-4ff6-b149-08d66fcd8bbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:54.457552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f5aa1f-7143-4ff6-b149-08d66fcd8bbf None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:54.463441 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 647/1287] 10.222.0.47 () {56 vars in 1374 bytes} [Thu Jun 30 10:17:54 2022] GET /identity/v3/role_assignments?user.id=46c99f43a9344d7d92632990e45bb9cf&scope.project.id=50902e0539434c82b505dc5009aeaf85&role.id=e63bda042e104a4a93fba15ffcfe1183 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:56.532759 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96536ab5-f8df-4f32-868a-c327c1b3ad6a None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:56.533206 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96536ab5-f8df-4f32-868a-c327c1b3ad6a None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:56.533323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96536ab5-f8df-4f32-868a-c327c1b3ad6a None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:56.534681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 641/1288] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:17:56.539975 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89231de-1b6a-48f6-82a9-c03f8a8f7705 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:56.540429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89231de-1b6a-48f6-82a9-c03f8a8f7705 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:56.540520 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89231de-1b6a-48f6-82a9-c03f8a8f7705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:56.558905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f89231de-1b6a-48f6-82a9-c03f8a8f7705 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:17:56.560190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f89231de-1b6a-48f6-82a9-c03f8a8f7705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:17:56.590205 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:56.625868 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 648/1289] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:17:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:17:56.648558 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96536ab5-f8df-4f32-868a-c327c1b3ad6a None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:56.748710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96536ab5-f8df-4f32-868a-c327c1b3ad6a None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:56.759531 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:56.781965 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2383c499-8397-4cb2-8d6d-e36618b0f237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:56.782764 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383c499-8397-4cb2-8d6d-e36618b0f237 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:56.783049 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383c499-8397-4cb2-8d6d-e36618b0f237 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:56.783381 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383c499-8397-4cb2-8d6d-e36618b0f237 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:56.784037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2383c499-8397-4cb2-8d6d-e36618b0f237 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:56.785407 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2383c499-8397-4cb2-8d6d-e36618b0f237 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:56.797321 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-2383c499-8397-4cb2-8d6d-e36618b0f237 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 10:17:56.797321 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:56.797321 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 10:17:56.797321 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 10:17:56.797321 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 10:17:56.797321 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 10:17:56.797321 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 10:17:56.797321 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:17:56.797321 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 10:17:56.797321 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:17:56.797321 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:17:56.797321 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:17:56.797321 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:17:56.797321 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:17:56.797321 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:17:56.797321 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:17:56.797321 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:17:56.800387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:17:56.800387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:17:56.800387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:17:56.800387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:17:56.800387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:17:56.800387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:17:56.800387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 10:17:56.800387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 10:17:56.800387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 10:17:56.800387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 10:17:56.800387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:17:56.800387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:17:56.800387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:17:56.800387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:17:56.800387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:17:56.800387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:17:56.801865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:17:56.801865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:17:56.801865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:17:56.801865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:17:56.801865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:17:56.801865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:17:56.801865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:17:56.801865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:17:56.801865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:17:56.801865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:17:56.801865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 10:17:56.801865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:17:56.801865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 10:17:56.801865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:17:56.801865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 642/1290] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jun 30 10:17:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:17:56.805280 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89231de-1b6a-48f6-82a9-c03f8a8f7705 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:56.909312 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89231de-1b6a-48f6-82a9-c03f8a8f7705 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:56.943433 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292234f7-9eb9-4e88-b4f4-254851add347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:56.944087 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292234f7-9eb9-4e88-b4f4-254851add347 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:56.944367 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292234f7-9eb9-4e88-b4f4-254851add347 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:56.944639 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292234f7-9eb9-4e88-b4f4-254851add347 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:56.945333 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292234f7-9eb9-4e88-b4f4-254851add347 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:56.945634 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292234f7-9eb9-4e88-b4f4-254851add347 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:56.947019 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292234f7-9eb9-4e88-b4f4-254851add347 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:56.951899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 649/1291] 10.222.0.47 () {56 vars in 1116 bytes} [Thu Jun 30 10:17:56 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:56.957822 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2383c499-8397-4cb2-8d6d-e36618b0f237 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:57.057853 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2383c499-8397-4cb2-8d6d-e36618b0f237 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:57.090388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0c109-1c8a-4348-9220-ae4911826318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:57.091046 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0c109-1c8a-4348-9220-ae4911826318 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:57.091355 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0c109-1c8a-4348-9220-ae4911826318 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:57.091631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0c109-1c8a-4348-9220-ae4911826318 None admin] PATH_INFO: `/v3/services` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:57.092325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0c109-1c8a-4348-9220-ae4911826318 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:17:57.092627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0c109-1c8a-4348-9220-ae4911826318 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:57.093983 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0c109-1c8a-4348-9220-ae4911826318 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:57.099045 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 643/1292] 10.222.0.47 () {56 vars in 1116 bytes} [Thu Jun 30 10:17:56 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:17:59.180865 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c8c48-d57a-4318-ab2b-cf3686edcf57 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:59.181400 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c8c48-d57a-4318-ab2b-cf3686edcf57 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:59.181400 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2c8c48-d57a-4318-ab2b-cf3686edcf57 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:59.182807 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 650/1293] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:17:59.188177 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b201e-c9bc-4e6e-97fb-9d04390c151c None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:59.188654 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b201e-c9bc-4e6e-97fb-9d04390c151c None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:59.188744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b201e-c9bc-4e6e-97fb-9d04390c151c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:59.206080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-563b201e-c9bc-4e6e-97fb-9d04390c151c None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:17:59.207344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-563b201e-c9bc-4e6e-97fb-9d04390c151c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:17:59.238526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:59.277499 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 644/1294] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:17:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:17:59.301063 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c8c48-d57a-4318-ab2b-cf3686edcf57 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:17:59.333431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:17:59.403090 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2c8c48-d57a-4318-ab2b-cf3686edcf57 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:17:59.436361 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2fb8d-2c4b-4288-bfde-14f36057f364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:17:59.436961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2fb8d-2c4b-4288-bfde-14f36057f364 None admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:17:59.437246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2fb8d-2c4b-4288-bfde-14f36057f364 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:17:59.437525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2fb8d-2c4b-4288-bfde-14f36057f364 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:17:59.438243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2fb8d-2c4b-4288-bfde-14f36057f364 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:17:59.439644 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2fb8d-2c4b-4288-bfde-14f36057f364 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:17:59.566353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 651/1295] 10.222.0.47 () {60 vars in 1141 bytes} [Thu Jun 30 10:17:59 2022] POST /identity/v3/services => generated 246 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:18:01.592676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f0260-152e-46f4-a279-b0421335decb None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:18:01.593259 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f0260-152e-46f4-a279-b0421335decb None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:18:01.593259 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f0260-152e-46f4-a279-b0421335decb None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:18:01.594612 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 645/1296] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:18:01.600302 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431ec85-0699-465b-ad54-b5d8e54c8c26 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:18:01.600725 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431ec85-0699-465b-ad54-b5d8e54c8c26 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:18:01.600861 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e431ec85-0699-465b-ad54-b5d8e54c8c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:18:01.617738 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e431ec85-0699-465b-ad54-b5d8e54c8c26 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:18:01.618929 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e431ec85-0699-465b-ad54-b5d8e54c8c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:18:01.691192 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 652/1297] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:18:01 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:18:01.714188 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f0260-152e-46f4-a279-b0421335decb None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:18:01.813754 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f0260-152e-46f4-a279-b0421335decb None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:18:01.850813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7ee9c-b9ab-4cb5-bdf9-56ca8a85af7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:18:01.851826 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7ee9c-b9ab-4cb5-bdf9-56ca8a85af7a None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:18:01.852118 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7ee9c-b9ab-4cb5-bdf9-56ca8a85af7a None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:18:01.852396 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e7ee9c-b9ab-4cb5-bdf9-56ca8a85af7a None admin] PATH_INFO: `/v3/services/placement` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:18:01.853056 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e7ee9c-b9ab-4cb5-bdf9-56ca8a85af7a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:18:01.854435 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e7ee9c-b9ab-4cb5-bdf9-56ca8a85af7a None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:18:01.866561 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-d3e7ee9c-b9ab-4cb5-bdf9-56ca8a85af7a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 10:18:01.866561 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:18:01.866561 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 10:18:01.866561 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 10:18:01.866561 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 10:18:01.866561 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 10:18:01.866561 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 10:18:01.866561 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:18:01.866561 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 10:18:01.866561 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:18:01.866561 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:18:01.866561 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:18:01.866561 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:18:01.866561 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:18:01.866561 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:18:01.866561 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:18:01.866561 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:18:01.868003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:18:01.868003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:18:01.868003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:18:01.868003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:18:01.868003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:18:01.868003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:18:01.868003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 10:18:01.868003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 10:18:01.868003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 10:18:01.868003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 10:18:01.868003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:18:01.868003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:18:01.868003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:18:01.868003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:18:01.868003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:18:01.868003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:18:01.869314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:18:01.869314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:18:01.869314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:18:01.869314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:18:01.869314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:18:01.869314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:18:01.869314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:18:01.869314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:18:01.869314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:18:01.869314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:18:01.869314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 10:18:01.869314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:18:01.869314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 10:18:01.869314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:18:01.869314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 646/1298] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jun 30 10:18:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:18:01.874062 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431ec85-0699-465b-ad54-b5d8e54c8c26 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:18:01.892230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:18:01.970656 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e431ec85-0699-465b-ad54-b5d8e54c8c26 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:18:02.003613 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2ec5d-1d51-43e5-931d-ef80683ff18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:18:02.004358 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2ec5d-1d51-43e5-931d-ef80683ff18e None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:18:02.004867 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2ec5d-1d51-43e5-931d-ef80683ff18e None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:18:02.005333 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff2ec5d-1d51-43e5-931d-ef80683ff18e None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:18:02.006672 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff2ec5d-1d51-43e5-931d-ef80683ff18e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:18:02.006985 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff2ec5d-1d51-43e5-931d-ef80683ff18e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:18:02.008456 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff2ec5d-1d51-43e5-931d-ef80683ff18e None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:18:02.013483 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 653/1299] 10.222.0.47 () {56 vars in 1116 bytes} [Thu Jun 30 10:18:01 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:18:02.019425 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7ee9c-b9ab-4cb5-bdf9-56ca8a85af7a None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:18:02.051808 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:18:02.118779 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e7ee9c-b9ab-4cb5-bdf9-56ca8a85af7a None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:18:02.151883 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe569b8-0bc1-46a0-bf70-c0fa7dc9837e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:18:02.152541 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe569b8-0bc1-46a0-bf70-c0fa7dc9837e None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:18:02.152828 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe569b8-0bc1-46a0-bf70-c0fa7dc9837e None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:18:02.153107 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe569b8-0bc1-46a0-bf70-c0fa7dc9837e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:18:02.153900 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe569b8-0bc1-46a0-bf70-c0fa7dc9837e None admin] RBAC: Adding query filter params (interface=public, service_id=48fa19b045264a46a26962976b19aa33, region_id=RegionOne) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:18:02.154214 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe569b8-0bc1-46a0-bf70-c0fa7dc9837e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:18:02.155616 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe569b8-0bc1-46a0-bf70-c0fa7dc9837e None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:18:02.161389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 647/1300] 10.222.0.47 () {56 vars in 1251 bytes} [Thu Jun 30 10:18:02 2022] GET /identity/v3/endpoints?service_id=48fa19b045264a46a26962976b19aa33&interface=public®ion_id=RegionOne => generated 194 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:18:02.167481 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2ec5d-1d51-43e5-931d-ef80683ff18e None admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:18:02.251965 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:18:02.269714 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff2ec5d-1d51-43e5-931d-ef80683ff18e None admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:18:02.302568 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab2963b-5d89-43e4-b15e-8e82d8abcfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:18:02.303227 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2963b-5d89-43e4-b15e-8e82d8abcfee None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:18:02.303537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2963b-5d89-43e4-b15e-8e82d8abcfee None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:18:02.303817 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab2963b-5d89-43e4-b15e-8e82d8abcfee None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:18:02.304501 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2963b-5d89-43e4-b15e-8e82d8abcfee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:18:02.305884 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab2963b-5d89-43e4-b15e-8e82d8abcfee None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:18:02.312556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 654/1301] 10.222.0.47 () {56 vars in 1087 bytes} [Thu Jun 30 10:18:02 2022] GET /identity/v3/services => generated 2191 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:18:04.340144 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243aa8f-a8fb-4391-8f22-9eb4319351e8 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:18:04.340707 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243aa8f-a8fb-4391-8f22-9eb4319351e8 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:18:04.340707 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f243aa8f-a8fb-4391-8f22-9eb4319351e8 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:18:04.342084 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 648/1302] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:18:04.347204 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7fbc3-220e-436a-bb6e-880fe96960c7 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:18:04.347611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7fbc3-220e-436a-bb6e-880fe96960c7 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:18:04.348589 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7fbc3-220e-436a-bb6e-880fe96960c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:18:04.366093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-17f7fbc3-220e-436a-bb6e-880fe96960c7 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:18:04.367288 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-17f7fbc3-220e-436a-bb6e-880fe96960c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:18:04.434701 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 655/1303] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:18:04 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 30 10:18:04.458223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243aa8f-a8fb-4391-8f22-9eb4319351e8 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:18:04.552441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:18:04.559641 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f243aa8f-a8fb-4391-8f22-9eb4319351e8 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:18:04.592507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b65d3-cc11-47a2-b13f-1f22fdb79ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:18:04.593245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b65d3-cc11-47a2-b13f-1f22fdb79ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:18:04.593531 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b65d3-cc11-47a2-b13f-1f22fdb79ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:18:04.593808 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1b65d3-cc11-47a2-b13f-1f22fdb79ad8 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:18:04.594467 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b65d3-cc11-47a2-b13f-1f22fdb79ad8 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:18:04.595889 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1b65d3-cc11-47a2-b13f-1f22fdb79ad8 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:18:04.608168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application [None req-6f1b65d3-cc11-47a2-b13f-1f22fdb79ad8 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 10:18:04.608168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:18:04.608168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 30 10:18:04.608168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 30 10:18:04.608168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 30 10:18:04.608168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 30 10:18:04.608168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 30 10:18:04.608168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:18:04.608168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 10:18:04.608168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:18:04.608168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 30 10:18:04.608168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:18:04.608168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 30 10:18:04.608168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 30 10:18:04.608168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 30 10:18:04.608168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 30 10:18:04.608168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 30 10:18:04.609376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 30 10:18:04.609376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 30 10:18:04.609376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 30 10:18:04.609376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 30 10:18:04.609376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 30 10:18:04.609376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 30 10:18:04.609376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 30 10:18:04.609376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 30 10:18:04.609376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 30 10:18:04.609376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 30 10:18:04.609376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 30 10:18:04.609376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 30 10:18:04.609376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 30 10:18:04.609376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 30 10:18:04.609376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 30 10:18:04.609376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.get_or_create( Jun 30 10:18:04.610573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 30 10:18:04.610573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application with Lock( Jun 30 10:18:04.610573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 30 10:18:04.610573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self._enter() Jun 30 10:18:04.610573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 30 10:18:04.610573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 30 10:18:04.610573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 30 10:18:04.610573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application return self.creator() Jun 30 10:18:04.610573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 30 10:18:04.610573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application created_value = creator( Jun 30 10:18:04.610573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 30 10:18:04.610573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 30 10:18:04.610573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 30 10:18:04.610573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: ERROR keystone.server.flask.application Jun 30 10:18:04.610573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 649/1304] 10.222.0.47 () {56 vars in 1117 bytes} [Thu Jun 30 10:18:04 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 30 10:18:04.616130 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7fbc3-220e-436a-bb6e-880fe96960c7 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:18:04.688817 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:18:04.710424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7fbc3-220e-436a-bb6e-880fe96960c7 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:18:04.743948 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb57a033-8723-4c6b-bfe6-3e2fd81fbc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:18:04.744599 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57a033-8723-4c6b-bfe6-3e2fd81fbc82 None admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:18:04.744882 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57a033-8723-4c6b-bfe6-3e2fd81fbc82 None admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:18:04.745155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb57a033-8723-4c6b-bfe6-3e2fd81fbc82 None admin] PATH_INFO: `/v3/services` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:18:04.745850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb57a033-8723-4c6b-bfe6-3e2fd81fbc82 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:18:04.746157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb57a033-8723-4c6b-bfe6-3e2fd81fbc82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:18:04.747542 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb57a033-8723-4c6b-bfe6-3e2fd81fbc82 None admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:18:04.752549 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 656/1305] 10.222.0.47 () {56 vars in 1116 bytes} [Thu Jun 30 10:18:04 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:18:04.758763 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b65d3-cc11-47a2-b13f-1f22fdb79ad8 None admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:18:04.861396 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1b65d3-cc11-47a2-b13f-1f22fdb79ad8 None admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:18:04.895734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4529ca7-d65b-44a7-91ba-6891acca0134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:18:04.898862 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4529ca7-d65b-44a7-91ba-6891acca0134 None admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:18:04.898862 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4529ca7-d65b-44a7-91ba-6891acca0134 None admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:18:04.898862 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4529ca7-d65b-44a7-91ba-6891acca0134 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:18:04.898862 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4529ca7-d65b-44a7-91ba-6891acca0134 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:18:04.899240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4529ca7-d65b-44a7-91ba-6891acca0134 None admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:18:04.963519 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 650/1306] 10.222.0.47 () {60 vars in 1144 bytes} [Thu Jun 30 10:18:04 2022] POST /identity/v3/endpoints => generated 331 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:18:14.749573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=30 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:18:14.958967 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:25:00.771071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516cc85-9301-4f51-b364-5e3c23d5f965 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:00.775760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516cc85-9301-4f51-b364-5e3c23d5f965 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:00.775760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516cc85-9301-4f51-b364-5e3c23d5f965 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:00.777545 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 657/1307] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:25:00 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:25:00.779002 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9508a39-26e6-413c-a87f-fccfa4b2a307 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:00.780220 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9508a39-26e6-413c-a87f-fccfa4b2a307 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:00.780812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9508a39-26e6-413c-a87f-fccfa4b2a307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:00.889597 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-f9508a39-26e6-413c-a87f-fccfa4b2a307 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:25:00.891359 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-f9508a39-26e6-413c-a87f-fccfa4b2a307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:01.009880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 651/1308] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:25:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:25:01.065149 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984891f-6fc7-400a-b434-58de2e7d3d4a None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:01.065475 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984891f-6fc7-400a-b434-58de2e7d3d4a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:01.065715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5984891f-6fc7-400a-b434-58de2e7d3d4a None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:01.067255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 658/1309] 10.222.0.47 () {54 vars in 964 bytes} [Thu Jun 30 10:25:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:25:01.074461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3826da-da50-4d2b-8c51-a96440b98a6c None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:01.074753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3826da-da50-4d2b-8c51-a96440b98a6c None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:01.075221 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3826da-da50-4d2b-8c51-a96440b98a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:01.160475 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-4e3826da-da50-4d2b-8c51-a96440b98a6c None None] MFA Rules not processed for user `059b05007c0144f7a50dbd9f0ef41359`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:25:01.163754 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-4e3826da-da50-4d2b-8c51-a96440b98a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:01.187084 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:25:01.245412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 652/1310] 10.222.0.47 () {58 vars in 1061 bytes} [Thu Jun 30 10:25:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:25:01.254897 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984891f-6fc7-400a-b434-58de2e7d3d4a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:25:01.256696 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5984891f-6fc7-400a-b434-58de2e7d3d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:01.374715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5984891f-6fc7-400a-b434-58de2e7d3d4a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:25:01.457513 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0240fc-7e28-452a-89a8-25a67b5799bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:25:01.458090 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0240fc-7e28-452a-89a8-25a67b5799bb service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:01.458374 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0240fc-7e28-452a-89a8-25a67b5799bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:01.458652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0240fc-7e28-452a-89a8-25a67b5799bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:01.460490 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ac0240fc-7e28-452a-89a8-25a67b5799bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:01.527436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:25:01.584845 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0240fc-7e28-452a-89a8-25a67b5799bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:25:01.586302 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0240fc-7e28-452a-89a8-25a67b5799bb service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:25:01.606519 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 659/1311] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:25:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:25:03.997516 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b9aea-f751-4d6d-9f97-ff80b2588978 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:03.998053 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b9aea-f751-4d6d-9f97-ff80b2588978 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:03.998053 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b9aea-f751-4d6d-9f97-ff80b2588978 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:03.999654 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 653/1312] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:25:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:25:04.004589 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bed30-c3f0-4a5e-8499-fade68f365db None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:04.005006 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bed30-c3f0-4a5e-8499-fade68f365db None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:04.005222 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015bed30-c3f0-4a5e-8499-fade68f365db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:04.080267 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-015bed30-c3f0-4a5e-8499-fade68f365db None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:25:04.081500 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-015bed30-c3f0-4a5e-8499-fade68f365db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:04.198987 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 660/1313] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:25:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:25:04.235920 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b9aea-f751-4d6d-9f97-ff80b2588978 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:25:04.237541 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-414b9aea-f751-4d6d-9f97-ff80b2588978 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:04.354852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b9aea-f751-4d6d-9f97-ff80b2588978 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:25:04.438046 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:25:04.441352 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70131d90-f150-418d-9f5a-9edfde3ace02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:25:04.442167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70131d90-f150-418d-9f5a-9edfde3ace02 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:04.442679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70131d90-f150-418d-9f5a-9edfde3ace02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:04.443864 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70131d90-f150-418d-9f5a-9edfde3ace02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:04.446475 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-70131d90-f150-418d-9f5a-9edfde3ace02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:04.564208 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70131d90-f150-418d-9f5a-9edfde3ace02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:25:04.565667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70131d90-f150-418d-9f5a-9edfde3ace02 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:25:04.584933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 654/1314] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:25:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:25:07.102560 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b486a-0551-476e-a104-eac20f795698 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:07.103137 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b486a-0551-476e-a104-eac20f795698 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:07.103137 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0b486a-0551-476e-a104-eac20f795698 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:07.104513 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 661/1315] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:25:07.109561 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486c17f-22ad-4406-b397-e1173ea2a722 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:07.109972 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486c17f-22ad-4406-b397-e1173ea2a722 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:07.110125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e486c17f-22ad-4406-b397-e1173ea2a722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:07.173708 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-e486c17f-22ad-4406-b397-e1173ea2a722 None None] MFA Rules not processed for user `db5cbfa9be364e0d9c75eac6090e7967`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:25:07.174912 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-e486c17f-22ad-4406-b397-e1173ea2a722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:07.226193 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:25:07.268356 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 655/1316] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:25:07 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:25:07.308383 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b486a-0551-476e-a104-eac20f795698 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:25:07.309909 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ca0b486a-0551-476e-a104-eac20f795698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:07.319517 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:25:07.412794 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0b486a-0551-476e-a104-eac20f795698 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:25:07.485832 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8047b7-02a4-4535-9b3e-795afa31be9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:25:07.486484 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8047b7-02a4-4535-9b3e-795afa31be9e service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:07.486771 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8047b7-02a4-4535-9b3e-795afa31be9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:07.487045 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8047b7-02a4-4535-9b3e-795afa31be9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:07.488722 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1d8047b7-02a4-4535-9b3e-795afa31be9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:07.560696 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:25:07.608846 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8047b7-02a4-4535-9b3e-795afa31be9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:25:07.610375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8047b7-02a4-4535-9b3e-795afa31be9e service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:25:07.629111 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 662/1317] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:25:07 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:25:10.461169 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9f672-4ce7-4848-8aea-3c9c1bc2bed1 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:10.461712 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9f672-4ce7-4848-8aea-3c9c1bc2bed1 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:10.461712 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9f672-4ce7-4848-8aea-3c9c1bc2bed1 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:10.463093 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 656/1318] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:25:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:25:10.468346 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8c4d5-f103-48d8-b5a5-d7e712577df6 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:10.468739 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8c4d5-f103-48d8-b5a5-d7e712577df6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:10.468909 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8c4d5-f103-48d8-b5a5-d7e712577df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:10.540839 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-07c8c4d5-f103-48d8-b5a5-d7e712577df6 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:25:10.542103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-07c8c4d5-f103-48d8-b5a5-d7e712577df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:10.643629 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 663/1319] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:25:10 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:25:10.682907 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9f672-4ce7-4848-8aea-3c9c1bc2bed1 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:25:10.684482 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-c4c9f672-4ce7-4848-8aea-3c9c1bc2bed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:10.786207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9f672-4ce7-4848-8aea-3c9c1bc2bed1 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:25:10.863713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19c6e20-5178-47e1-9461-3ad654bae437 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:25:10.864366 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c6e20-5178-47e1-9461-3ad654bae437 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:10.865514 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c6e20-5178-47e1-9461-3ad654bae437 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:10.865955 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19c6e20-5178-47e1-9461-3ad654bae437 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:10.867670 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-b19c6e20-5178-47e1-9461-3ad654bae437 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:10.874576 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:25:10.977915 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c6e20-5178-47e1-9461-3ad654bae437 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:25:10.979376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19c6e20-5178-47e1-9461-3ad654bae437 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:25:10.996936 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 657/1320] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:25:10 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:25:14.034265 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08a42a-2d3d-47fc-bbaf-075cd9c1fba5 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:14.034807 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08a42a-2d3d-47fc-bbaf-075cd9c1fba5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:14.034807 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f08a42a-2d3d-47fc-bbaf-075cd9c1fba5 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:14.036241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 664/1321] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:25:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:25:14.041869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264c0cf-4d63-414d-ab34-479a039671d9 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:14.042269 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264c0cf-4d63-414d-ab34-479a039671d9 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:14.042431 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1264c0cf-4d63-414d-ab34-479a039671d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:14.107546 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-1264c0cf-4d63-414d-ab34-479a039671d9 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:25:14.108759 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-1264c0cf-4d63-414d-ab34-479a039671d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:14.172690 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:25:14.213098 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 658/1322] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:25:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:25:14.264484 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48575574-434e-4417-9624-145f884c1638 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:14.264915 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48575574-434e-4417-9624-145f884c1638 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:14.265283 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48575574-434e-4417-9624-145f884c1638 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:14.266786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 665/1323] 10.222.0.47 () {54 vars in 964 bytes} [Thu Jun 30 10:25:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:25:14.273716 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0383b9-3b23-4ab7-b95d-9c06c694f349 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:14.274098 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0383b9-3b23-4ab7-b95d-9c06c694f349 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:14.274376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0383b9-3b23-4ab7-b95d-9c06c694f349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:14.356507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-6d0383b9-3b23-4ab7-b95d-9c06c694f349 None None] MFA Rules not processed for user `059b05007c0144f7a50dbd9f0ef41359`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:25:14.357675 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-6d0383b9-3b23-4ab7-b95d-9c06c694f349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:14.444045 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 659/1324] 10.222.0.47 () {58 vars in 1061 bytes} [Thu Jun 30 10:25:14 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:25:14.453131 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48575574-434e-4417-9624-145f884c1638 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:25:14.454674 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-48575574-434e-4417-9624-145f884c1638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:14.474406 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:25:14.558893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48575574-434e-4417-9624-145f884c1638 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:25:14.632496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb4a8c5-aa05-46f1-8d34-8d1a93346ff6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:25:14.633154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4a8c5-aa05-46f1-8d34-8d1a93346ff6 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:14.633438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4a8c5-aa05-46f1-8d34-8d1a93346ff6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:14.633712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb4a8c5-aa05-46f1-8d34-8d1a93346ff6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:14.635390 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0eb4a8c5-aa05-46f1-8d34-8d1a93346ff6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:14.707426 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:25:14.746498 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4a8c5-aa05-46f1-8d34-8d1a93346ff6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:25:14.748020 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb4a8c5-aa05-46f1-8d34-8d1a93346ff6 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:25:14.767152 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 666/1325] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:25:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:25:17.068744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9f0c6-f22a-4d1c-9c33-4257db6c7197 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:17.069325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9f0c6-f22a-4d1c-9c33-4257db6c7197 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:17.069325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9f0c6-f22a-4d1c-9c33-4257db6c7197 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:17.070677 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 660/1326] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:25:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:25:17.076085 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1853af8-8e04-47e8-981b-0bd84b75a946 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:17.076490 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1853af8-8e04-47e8-981b-0bd84b75a946 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:17.076740 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1853af8-8e04-47e8-981b-0bd84b75a946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:17.142565 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d1853af8-8e04-47e8-981b-0bd84b75a946 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:25:17.143854 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d1853af8-8e04-47e8-981b-0bd84b75a946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:17.248192 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 667/1327] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:25:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:25:17.283604 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9f0c6-f22a-4d1c-9c33-4257db6c7197 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:25:17.285270 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-f1f9f0c6-f22a-4d1c-9c33-4257db6c7197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:17.352987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:25:17.397189 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9f0c6-f22a-4d1c-9c33-4257db6c7197 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:25:17.470047 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4646ecf-ceaf-4950-b4eb-f1bb8d397a76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:25:17.470702 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4646ecf-ceaf-4950-b4eb-f1bb8d397a76 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:17.470987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4646ecf-ceaf-4950-b4eb-f1bb8d397a76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:17.471258 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4646ecf-ceaf-4950-b4eb-f1bb8d397a76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:17.472939 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-a4646ecf-ceaf-4950-b4eb-f1bb8d397a76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:17.582523 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:25:17.585034 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4646ecf-ceaf-4950-b4eb-f1bb8d397a76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:25:17.586474 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4646ecf-ceaf-4950-b4eb-f1bb8d397a76 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:25:17.604505 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 661/1328] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:25:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:25:19.711197 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385b0d5-06bd-4a3c-9cb7-fc293a173b01 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:19.711798 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385b0d5-06bd-4a3c-9cb7-fc293a173b01 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:19.711798 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385b0d5-06bd-4a3c-9cb7-fc293a173b01 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:19.713328 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 668/1329] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:25:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:25:19.718442 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd948b-ebe5-4025-bbc7-997fde136678 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:19.718846 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd948b-ebe5-4025-bbc7-997fde136678 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:19.719124 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bd948b-ebe5-4025-bbc7-997fde136678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:19.787580 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-13bd948b-ebe5-4025-bbc7-997fde136678 None None] MFA Rules not processed for user `db5cbfa9be364e0d9c75eac6090e7967`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:25:19.788718 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-13bd948b-ebe5-4025-bbc7-997fde136678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:19.900343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 662/1330] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:25:19 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:25:19.938228 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385b0d5-06bd-4a3c-9cb7-fc293a173b01 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:25:19.939786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a385b0d5-06bd-4a3c-9cb7-fc293a173b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:19.983053 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:25:20.067174 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385b0d5-06bd-4a3c-9cb7-fc293a173b01 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:25:20.143653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5da9b-2eba-422f-b1b6-a70ea961d648 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:25:20.144303 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5da9b-2eba-422f-b1b6-a70ea961d648 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:20.144586 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5da9b-2eba-422f-b1b6-a70ea961d648 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:20.144862 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5da9b-2eba-422f-b1b6-a70ea961d648 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:20.146715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d0e5da9b-2eba-422f-b1b6-a70ea961d648 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:20.224350 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:25:20.257419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5da9b-2eba-422f-b1b6-a70ea961d648 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:25:20.258868 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5da9b-2eba-422f-b1b6-a70ea961d648 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:25:20.277546 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 669/1331] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:25:19 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:25:23.183966 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc2629-745b-43ce-a592-fe37ed3ddc5a None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:23.184485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc2629-745b-43ce-a592-fe37ed3ddc5a None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:23.184611 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc2629-745b-43ce-a592-fe37ed3ddc5a None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:23.185907 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 663/1332] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:25:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:25:23.191089 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df233191-c6e3-454c-a39c-56ac40af326d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:23.191557 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df233191-c6e3-454c-a39c-56ac40af326d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:23.191674 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df233191-c6e3-454c-a39c-56ac40af326d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:23.255922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-df233191-c6e3-454c-a39c-56ac40af326d None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:25:23.257122 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-df233191-c6e3-454c-a39c-56ac40af326d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:23.353464 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 670/1333] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:25:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:25:23.389548 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc2629-745b-43ce-a592-fe37ed3ddc5a None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:25:23.391100 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-65bc2629-745b-43ce-a592-fe37ed3ddc5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:23.460449 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:25:23.495140 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc2629-745b-43ce-a592-fe37ed3ddc5a None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:25:23.572130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8cc90f-72ba-49ea-a6ad-1c21e935f719 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:25:23.572767 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cc90f-72ba-49ea-a6ad-1c21e935f719 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:23.573045 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cc90f-72ba-49ea-a6ad-1c21e935f719 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:23.573317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8cc90f-72ba-49ea-a6ad-1c21e935f719 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:23.574949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-5d8cc90f-72ba-49ea-a6ad-1c21e935f719 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:23.682581 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8cc90f-72ba-49ea-a6ad-1c21e935f719 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:25:23.684043 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8cc90f-72ba-49ea-a6ad-1c21e935f719 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:25:23.694720 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:25:23.703502 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 664/1334] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:25:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:25:25.790268 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5838297-9288-493e-8d74-2df64ff74f83 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:25.790780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5838297-9288-493e-8d74-2df64ff74f83 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:25.790780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5838297-9288-493e-8d74-2df64ff74f83 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:25.792147 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 671/1335] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:25:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:25:25.797165 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1dd7a-b4e5-4f98-83aa-f474145268f4 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:25.797582 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1dd7a-b4e5-4f98-83aa-f474145268f4 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:25.797727 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb1dd7a-b4e5-4f98-83aa-f474145268f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:25.861464 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-4eb1dd7a-b4e5-4f98-83aa-f474145268f4 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:25:25.862664 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-4eb1dd7a-b4e5-4f98-83aa-f474145268f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:25.958513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 665/1336] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:25:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:25:25.999482 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5838297-9288-493e-8d74-2df64ff74f83 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:25:26.001115 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f5838297-9288-493e-8d74-2df64ff74f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:26.042235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:25:26.105758 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5838297-9288-493e-8d74-2df64ff74f83 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:25:26.178053 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1853370d-1302-481d-b844-39d0c4404439 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:25:26.178697 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853370d-1302-481d-b844-39d0c4404439 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:26.178980 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853370d-1302-481d-b844-39d0c4404439 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:26.179252 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1853370d-1302-481d-b844-39d0c4404439 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:26.180908 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1853370d-1302-481d-b844-39d0c4404439 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:26.262665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:25:26.288752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853370d-1302-481d-b844-39d0c4404439 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:25:26.290186 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1853370d-1302-481d-b844-39d0c4404439 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:25:26.309156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 672/1337] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:25:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:25:29.099752 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e9ec0-f60a-4129-bdf2-3ffdfef6d684 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:29.100325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e9ec0-f60a-4129-bdf2-3ffdfef6d684 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:29.100325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946e9ec0-f60a-4129-bdf2-3ffdfef6d684 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:29.101688 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 666/1338] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:25:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:25:29.106912 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae2390-2e94-4641-a969-9d9a354ae1a1 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:29.107448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae2390-2e94-4641-a969-9d9a354ae1a1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:29.107548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dae2390-2e94-4641-a969-9d9a354ae1a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:29.173434 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-2dae2390-2e94-4641-a969-9d9a354ae1a1 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:25:29.174937 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2dae2390-2e94-4641-a969-9d9a354ae1a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:29.278941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 673/1339] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:25:29 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:25:29.314596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e9ec0-f60a-4129-bdf2-3ffdfef6d684 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:25:29.316169 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-946e9ec0-f60a-4129-bdf2-3ffdfef6d684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:29.393310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:25:29.420823 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946e9ec0-f60a-4129-bdf2-3ffdfef6d684 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:25:29.493483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06281fa7-9df5-4cd5-acb8-9cb5e829e8b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:25:29.494148 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06281fa7-9df5-4cd5-acb8-9cb5e829e8b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:29.494454 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06281fa7-9df5-4cd5-acb8-9cb5e829e8b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:29.494733 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06281fa7-9df5-4cd5-acb8-9cb5e829e8b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:29.496544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-06281fa7-9df5-4cd5-acb8-9cb5e829e8b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:29.611887 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06281fa7-9df5-4cd5-acb8-9cb5e829e8b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:25:29.613521 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06281fa7-9df5-4cd5-acb8-9cb5e829e8b6 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:25:29.634583 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 667/1340] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:25:29 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:25:33.966075 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2b736-813a-4e82-b72b-60cadfd698ec None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:33.966638 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2b736-813a-4e82-b72b-60cadfd698ec None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:33.966638 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2b736-813a-4e82-b72b-60cadfd698ec None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:33.968241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 674/1341] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:25:33.974198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed397c2-9ef3-41ee-aa5f-28e5f2a4bba4 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:33.974887 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed397c2-9ef3-41ee-aa5f-28e5f2a4bba4 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:33.974887 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed397c2-9ef3-41ee-aa5f-28e5f2a4bba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:33.983364 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:25:34.051894 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-1ed397c2-9ef3-41ee-aa5f-28e5f2a4bba4 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:25:34.053774 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-1ed397c2-9ef3-41ee-aa5f-28e5f2a4bba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:34.162778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 668/1342] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:25:34.201752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2b736-813a-4e82-b72b-60cadfd698ec None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:25:34.204014 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-dcc2b736-813a-4e82-b72b-60cadfd698ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:34.264191 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:25:34.311583 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2b736-813a-4e82-b72b-60cadfd698ec None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:25:34.315087 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f245f-3b6b-4299-973f-5bf761fe3263 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:25:34.315759 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f245f-3b6b-4299-973f-5bf761fe3263 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:34.316044 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f245f-3b6b-4299-973f-5bf761fe3263 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:34.316318 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f245f-3b6b-4299-973f-5bf761fe3263 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:34.325702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f245f-3b6b-4299-973f-5bf761fe3263 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:25:34.327147 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f245f-3b6b-4299-973f-5bf761fe3263 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:25:34.336906 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 675/1343] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:25:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:25:37.437994 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d3bcc-29c6-445d-b10e-d06e4fa4a664 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:37.438584 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d3bcc-29c6-445d-b10e-d06e4fa4a664 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:37.438584 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9d3bcc-29c6-445d-b10e-d06e4fa4a664 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:37.439950 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 669/1344] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:25:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:25:37.445279 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba981229-3670-4765-90a7-a81fa1cf4acd None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:37.445699 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba981229-3670-4765-90a7-a81fa1cf4acd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:37.445811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba981229-3670-4765-90a7-a81fa1cf4acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:37.474598 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ba981229-3670-4765-90a7-a81fa1cf4acd None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:25:37.475900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ba981229-3670-4765-90a7-a81fa1cf4acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:37.561537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 676/1345] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:25:37 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:25:37.598861 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d3bcc-29c6-445d-b10e-d06e4fa4a664 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:25:37.601381 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-db9d3bcc-29c6-445d-b10e-d06e4fa4a664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:37.614713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9d3bcc-29c6-445d-b10e-d06e4fa4a664 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:25:37.618256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd1e92b-70f6-42f2-9639-3443f193ed3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:25:37.618890 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1e92b-70f6-42f2-9639-3443f193ed3d service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:37.619175 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1e92b-70f6-42f2-9639-3443f193ed3d service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:37.619485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd1e92b-70f6-42f2-9639-3443f193ed3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:37.628487 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd1e92b-70f6-42f2-9639-3443f193ed3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:25:37.629855 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd1e92b-70f6-42f2-9639-3443f193ed3d service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:25:37.639626 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 670/1346] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:25:37 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:25:42.332596 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6a4b6-80b5-48e0-a286-5e7b4018e762 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:42.333699 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6a4b6-80b5-48e0-a286-5e7b4018e762 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:42.333813 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b6a4b6-80b5-48e0-a286-5e7b4018e762 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:42.335365 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 677/1347] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:25:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:25:42.340616 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b69dab-3acd-419f-9719-91bc85f7f278 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:42.341022 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b69dab-3acd-419f-9719-91bc85f7f278 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:42.341162 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b69dab-3acd-419f-9719-91bc85f7f278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:42.362901 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-51b69dab-3acd-419f-9719-91bc85f7f278 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:25:42.364270 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-51b69dab-3acd-419f-9719-91bc85f7f278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:42.372822 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 671/1348] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:25:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:25:42.412262 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6a4b6-80b5-48e0-a286-5e7b4018e762 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:25:42.423197 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b6a4b6-80b5-48e0-a286-5e7b4018e762 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:25:42.426831 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c628a158-dd01-4bdf-b3ce-6bcd6df74d4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:25:42.427557 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628a158-dd01-4bdf-b3ce-6bcd6df74d4a service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:42.427868 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628a158-dd01-4bdf-b3ce-6bcd6df74d4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:42.428144 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c628a158-dd01-4bdf-b3ce-6bcd6df74d4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:42.437755 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c628a158-dd01-4bdf-b3ce-6bcd6df74d4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:25:42.439214 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c628a158-dd01-4bdf-b3ce-6bcd6df74d4a service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:25:42.449866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 678/1349] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:25:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:25:47.595610 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253da66-d508-493e-b7db-59bd13bfe049 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:47.596592 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253da66-d508-493e-b7db-59bd13bfe049 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:47.597464 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253da66-d508-493e-b7db-59bd13bfe049 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:47.599681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 672/1350] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jun 30 10:25:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:25:47.607329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aab3ef-c7b8-4d0c-a207-db1d0c0d6d93 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:47.607805 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aab3ef-c7b8-4d0c-a207-db1d0c0d6d93 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:47.607898 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aab3ef-c7b8-4d0c-a207-db1d0c0d6d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:47.720744 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b4aab3ef-c7b8-4d0c-a207-db1d0c0d6d93 None None] MFA Rules not processed for user `46c99f43a9344d7d92632990e45bb9cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:25:47.722265 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b4aab3ef-c7b8-4d0c-a207-db1d0c0d6d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:47.753922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:25:47.807336 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 679/1351] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jun 30 10:25:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:25:52.129223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e741f-3b50-4a4a-9163-ade60ad7f179 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:52.130530 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e741f-3b50-4a4a-9163-ade60ad7f179 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:52.130530 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5e741f-3b50-4a4a-9163-ade60ad7f179 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:52.132052 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 673/1352] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jun 30 10:25:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:25:52.137148 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f565b6-4195-4740-80b0-059861651ea9 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:52.137608 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f565b6-4195-4740-80b0-059861651ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:52.138014 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f565b6-4195-4740-80b0-059861651ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:52.174113 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-49f565b6-4195-4740-80b0-059861651ea9 None None] MFA Rules not processed for user `46c99f43a9344d7d92632990e45bb9cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:25:52.175492 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-49f565b6-4195-4740-80b0-059861651ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:52.185384 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 680/1353] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jun 30 10:25:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:25:52.356327 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=72 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:25:53.620112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d36dd-66c0-45df-978b-df11258269c2 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:53.620756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d36dd-66c0-45df-978b-df11258269c2 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:53.620756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d36dd-66c0-45df-978b-df11258269c2 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:53.622031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 674/1354] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jun 30 10:25:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:25:53.626856 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c8508-7573-4d58-be65-c8ae8f93bade None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:53.627310 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c8508-7573-4d58-be65-c8ae8f93bade None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:53.627578 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c8508-7573-4d58-be65-c8ae8f93bade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:53.646844 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7f4c8508-7573-4d58-be65-c8ae8f93bade None None] MFA Rules not processed for user `46c99f43a9344d7d92632990e45bb9cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:25:53.648054 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7f4c8508-7573-4d58-be65-c8ae8f93bade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:53.656794 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 681/1355] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jun 30 10:25:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:25:57.264166 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c2e01-bbc6-465f-8913-5c07ef1bdd0d None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:57.265111 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c2e01-bbc6-465f-8913-5c07ef1bdd0d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:57.265623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c2e01-bbc6-465f-8913-5c07ef1bdd0d None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:57.268696 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 675/1356] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:25:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:25:57.273435 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbdc93-3ae8-4b31-90c7-13ef4a4b21da None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:57.274338 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbdc93-3ae8-4b31-90c7-13ef4a4b21da None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:57.274847 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bbdc93-3ae8-4b31-90c7-13ef4a4b21da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:57.300086 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e5bbdc93-3ae8-4b31-90c7-13ef4a4b21da None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:25:57.302238 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e5bbdc93-3ae8-4b31-90c7-13ef4a4b21da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:57.311077 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 682/1357] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:25:57.366964 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b4b49-69c1-46f1-bc59-b78ad2fe4c4a None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:57.367690 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b4b49-69c1-46f1-bc59-b78ad2fe4c4a None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:57.368209 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300b4b49-69c1-46f1-bc59-b78ad2fe4c4a None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:57.370211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 676/1358] 10.222.0.47 () {54 vars in 961 bytes} [Thu Jun 30 10:25:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:25:57.378841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497755e4-d6ad-41e6-b0c4-434e89f213e6 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:57.380239 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497755e4-d6ad-41e6-b0c4-434e89f213e6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:57.380850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497755e4-d6ad-41e6-b0c4-434e89f213e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:57.433235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-497755e4-d6ad-41e6-b0c4-434e89f213e6 None None] MFA Rules not processed for user `71f6d7715e6f49ffb9fdf6e1307bbd51`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:25:57.434445 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-497755e4-d6ad-41e6-b0c4-434e89f213e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:57.528347 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 683/1359] 10.222.0.47 () {58 vars in 1058 bytes} [Thu Jun 30 10:25:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:25:57.537390 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b4b49-69c1-46f1-bc59-b78ad2fe4c4a None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:25:57.552749 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300b4b49-69c1-46f1-bc59-b78ad2fe4c4a None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:25:57.558093 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8494b808-1f35-441d-8560-d3e5b3547611 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:25:57.558732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494b808-1f35-441d-8560-d3e5b3547611 service nova] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:57.559012 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494b808-1f35-441d-8560-d3e5b3547611 service nova] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:57.559344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8494b808-1f35-441d-8560-d3e5b3547611 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:57.568955 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494b808-1f35-441d-8560-d3e5b3547611 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:25:57.570555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8494b808-1f35-441d-8560-d3e5b3547611 service nova] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:25:57.580360 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 677/1360] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:25:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:25:59.871988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707e886-1a85-4c83-99ca-701d45f5ef67 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:59.872540 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707e886-1a85-4c83-99ca-701d45f5ef67 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:59.872540 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707e886-1a85-4c83-99ca-701d45f5ef67 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:59.874114 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 684/1361] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:25:59.879077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e2f4c-bb9f-402e-bba6-f27e59225b89 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:59.880033 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e2f4c-bb9f-402e-bba6-f27e59225b89 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:59.880228 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7e2f4c-bb9f-402e-bba6-f27e59225b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:59.898613 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-5c7e2f4c-bb9f-402e-bba6-f27e59225b89 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:25:59.899856 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-5c7e2f4c-bb9f-402e-bba6-f27e59225b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:25:59.908147 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 678/1362] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:25:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:25:59.953114 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707e886-1a85-4c83-99ca-701d45f5ef67 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:25:59.971677 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707e886-1a85-4c83-99ca-701d45f5ef67 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:25:59.978736 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5583e8-50bf-4827-9988-af8e7c2b07c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:25:59.979668 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5583e8-50bf-4827-9988-af8e7c2b07c5 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:25:59.980063 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5583e8-50bf-4827-9988-af8e7c2b07c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:25:59.980488 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5583e8-50bf-4827-9988-af8e7c2b07c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:25:59.990322 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5583e8-50bf-4827-9988-af8e7c2b07c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:25:59.992088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5583e8-50bf-4827-9988-af8e7c2b07c5 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:26:00.002696 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 685/1363] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:25:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:26:02.511251 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e9388-ab7e-41c2-a018-6fe36d688ff5 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:02.511727 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e9388-ab7e-41c2-a018-6fe36d688ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:02.511854 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62e9388-ab7e-41c2-a018-6fe36d688ff5 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:02.513225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 679/1364] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:26:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:26:02.518479 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc668097-94d8-4d3b-bbc3-858fa5877122 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:02.519248 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc668097-94d8-4d3b-bbc3-858fa5877122 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:02.520315 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc668097-94d8-4d3b-bbc3-858fa5877122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:02.538777 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-dc668097-94d8-4d3b-bbc3-858fa5877122 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:02.540013 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-dc668097-94d8-4d3b-bbc3-858fa5877122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:02.548140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 686/1365] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:26:02.598604 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3db448-7f07-4557-ad25-ed5d3b45a700 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:02.598904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3db448-7f07-4557-ad25-ed5d3b45a700 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:02.599180 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3db448-7f07-4557-ad25-ed5d3b45a700 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:02.600581 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 680/1366] 10.222.0.47 () {54 vars in 961 bytes} [Thu Jun 30 10:26:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:26:02.608903 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52cdb1-fcae-4239-bb4e-12f090b47f91 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:02.611679 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52cdb1-fcae-4239-bb4e-12f090b47f91 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:02.611679 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52cdb1-fcae-4239-bb4e-12f090b47f91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:02.628537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8f52cdb1-fcae-4239-bb4e-12f090b47f91 None None] MFA Rules not processed for user `71f6d7715e6f49ffb9fdf6e1307bbd51`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:02.629941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8f52cdb1-fcae-4239-bb4e-12f090b47f91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:02.640758 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 687/1367] 10.222.0.47 () {58 vars in 1058 bytes} [Thu Jun 30 10:26:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:26:02.649320 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3db448-7f07-4557-ad25-ed5d3b45a700 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:26:02.665176 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3db448-7f07-4557-ad25-ed5d3b45a700 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:26:02.671105 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6c0b7-4b2a-4416-93b8-b013a148969c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:26:02.671786 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6c0b7-4b2a-4416-93b8-b013a148969c service nova] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:02.672075 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6c0b7-4b2a-4416-93b8-b013a148969c service nova] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:02.672359 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6c0b7-4b2a-4416-93b8-b013a148969c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:02.681358 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d6c0b7-4b2a-4416-93b8-b013a148969c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:26:02.682932 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d6c0b7-4b2a-4416-93b8-b013a148969c service nova] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:26:02.693214 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 681/1368] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:26:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:26:04.958400 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a7efc-78d9-47a9-9a56-f9474d38f3e8 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:04.959044 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a7efc-78d9-47a9-9a56-f9474d38f3e8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:04.959044 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a7efc-78d9-47a9-9a56-f9474d38f3e8 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:04.960242 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 688/1369] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:26:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:26:04.965577 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615056d-d694-42fb-aa40-0b58b950a427 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:04.965989 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615056d-d694-42fb-aa40-0b58b950a427 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:04.966136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5615056d-d694-42fb-aa40-0b58b950a427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:04.984924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-5615056d-d694-42fb-aa40-0b58b950a427 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:04.986127 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-5615056d-d694-42fb-aa40-0b58b950a427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:04.994828 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 682/1370] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:26:05.039101 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a7efc-78d9-47a9-9a56-f9474d38f3e8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:26:05.054771 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a7efc-78d9-47a9-9a56-f9474d38f3e8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:26:05.060293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70059ad2-2e47-46f6-9db0-ec75f916f2bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:26:05.061055 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70059ad2-2e47-46f6-9db0-ec75f916f2bc service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:05.061434 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70059ad2-2e47-46f6-9db0-ec75f916f2bc service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:05.061843 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70059ad2-2e47-46f6-9db0-ec75f916f2bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:05.070639 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70059ad2-2e47-46f6-9db0-ec75f916f2bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:26:05.073648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70059ad2-2e47-46f6-9db0-ec75f916f2bc service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:26:05.083820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 689/1371] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:26:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:26:05.462808 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176af76-cf37-426c-a841-89f35a21ba68 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:05.463098 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176af76-cf37-426c-a841-89f35a21ba68 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:05.463617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7176af76-cf37-426c-a841-89f35a21ba68 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:05.465004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 683/1372] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:26:05.470455 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b54f76-2c2f-4ec3-9f9b-3962e2ddad8c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:05.470953 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b54f76-2c2f-4ec3-9f9b-3962e2ddad8c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:05.471414 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b54f76-2c2f-4ec3-9f9b-3962e2ddad8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:05.493774 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c5b54f76-2c2f-4ec3-9f9b-3962e2ddad8c None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:05.495325 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c5b54f76-2c2f-4ec3-9f9b-3962e2ddad8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:05.511321 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 690/1373] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:26:05.854582 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492592d-080f-439f-a819-49e64629c808 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:05.854835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492592d-080f-439f-a819-49e64629c808 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:05.855113 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492592d-080f-439f-a819-49e64629c808 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:05.856715 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 684/1374] 10.222.0.47 () {54 vars in 963 bytes} [Thu Jun 30 10:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:26:05.861718 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f44f7f-ebf6-4a90-bad6-5e470852d07a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:05.862125 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f44f7f-ebf6-4a90-bad6-5e470852d07a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:05.862508 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f44f7f-ebf6-4a90-bad6-5e470852d07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:05.869563 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:26:05.915072 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-24f44f7f-ebf6-4a90-bad6-5e470852d07a None None] MFA Rules not processed for user `07ab12e5d1d94a009c4d414e5c43ac92`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:05.916506 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-24f44f7f-ebf6-4a90-bad6-5e470852d07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:05.981521 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 691/1375] 10.222.0.47 () {58 vars in 1060 bytes} [Thu Jun 30 10:26:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:26:05.989581 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492592d-080f-439f-a819-49e64629c808 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:26:06.000362 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492592d-080f-439f-a819-49e64629c808 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:26:06.004728 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f013a-b5ad-4085-ad36-02b0fb65dbc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:26:06.005435 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f013a-b5ad-4085-ad36-02b0fb65dbc2 service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:06.005818 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f013a-b5ad-4085-ad36-02b0fb65dbc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:06.006200 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f013a-b5ad-4085-ad36-02b0fb65dbc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:06.016713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f013a-b5ad-4085-ad36-02b0fb65dbc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:26:06.018267 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f013a-b5ad-4085-ad36-02b0fb65dbc2 service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:26:06.029235 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 685/1376] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:26:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:26:07.267122 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7b5e0-2ece-4633-ad01-3670b005f033 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:07.267720 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7b5e0-2ece-4633-ad01-3670b005f033 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:07.267720 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb7b5e0-2ece-4633-ad01-3670b005f033 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:07.269097 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 692/1377] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:26:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:26:07.275130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf050764-3dfa-4979-9eb9-d9fcb7b06411 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:07.275581 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf050764-3dfa-4979-9eb9-d9fcb7b06411 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:07.275767 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf050764-3dfa-4979-9eb9-d9fcb7b06411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:07.293507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-bf050764-3dfa-4979-9eb9-d9fcb7b06411 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:07.294698 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-bf050764-3dfa-4979-9eb9-d9fcb7b06411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:07.302823 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 686/1378] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:26:07.347171 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7b5e0-2ece-4633-ad01-3670b005f033 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:26:07.361729 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb7b5e0-2ece-4633-ad01-3670b005f033 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:26:07.367656 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ea3316-4a29-4a14-b36c-b525cc48a8e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:26:07.368295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea3316-4a29-4a14-b36c-b525cc48a8e6 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:07.368577 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea3316-4a29-4a14-b36c-b525cc48a8e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:07.368850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ea3316-4a29-4a14-b36c-b525cc48a8e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:07.377357 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea3316-4a29-4a14-b36c-b525cc48a8e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:26:07.379471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ea3316-4a29-4a14-b36c-b525cc48a8e6 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:26:07.389523 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 693/1379] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:26:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:26:09.638234 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bf12b-c7f5-45d8-bbc2-e2254698bd08 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:09.638698 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bf12b-c7f5-45d8-bbc2-e2254698bd08 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:09.638808 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0bf12b-c7f5-45d8-bbc2-e2254698bd08 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:09.644252 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 687/1380] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:26:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:26:09.652962 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e27078-03cd-4ba4-b0db-4bb967db4114 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:09.653434 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e27078-03cd-4ba4-b0db-4bb967db4114 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:09.653529 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e27078-03cd-4ba4-b0db-4bb967db4114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:09.671953 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-86e27078-03cd-4ba4-b0db-4bb967db4114 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:09.673145 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-86e27078-03cd-4ba4-b0db-4bb967db4114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:09.681812 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 694/1381] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:26:09.731777 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0bf12b-c7f5-45d8-bbc2-e2254698bd08 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:26:09.749940 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0bf12b-c7f5-45d8-bbc2-e2254698bd08 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:26:09.759346 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845360b5-1a6c-43e6-b5ea-c94d847b6af2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:26:09.761210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845360b5-1a6c-43e6-b5ea-c94d847b6af2 service nova] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:09.761959 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845360b5-1a6c-43e6-b5ea-c94d847b6af2 service nova] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:09.762373 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845360b5-1a6c-43e6-b5ea-c94d847b6af2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:09.775057 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845360b5-1a6c-43e6-b5ea-c94d847b6af2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:26:09.776906 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845360b5-1a6c-43e6-b5ea-c94d847b6af2 service nova] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:26:09.799461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 688/1382] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:26:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:26:12.007848 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead118c9-abab-4263-a059-443a35c4a782 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:12.008399 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead118c9-abab-4263-a059-443a35c4a782 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:12.008399 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead118c9-abab-4263-a059-443a35c4a782 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:12.009767 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 695/1383] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:26:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:26:12.015247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc105e7-4705-4890-8402-e4c19d78f5d9 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:12.015691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc105e7-4705-4890-8402-e4c19d78f5d9 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:12.015868 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc105e7-4705-4890-8402-e4c19d78f5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:12.035628 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-4dc105e7-4705-4890-8402-e4c19d78f5d9 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:12.036810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-4dc105e7-4705-4890-8402-e4c19d78f5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:12.044935 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 689/1384] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:26:12.088183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead118c9-abab-4263-a059-443a35c4a782 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:26:12.104384 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead118c9-abab-4263-a059-443a35c4a782 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:26:12.110108 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58a7f1-bc86-472f-be25-c454eead747f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:26:12.110871 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58a7f1-bc86-472f-be25-c454eead747f service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:12.111224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58a7f1-bc86-472f-be25-c454eead747f service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:12.111607 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58a7f1-bc86-472f-be25-c454eead747f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:12.121598 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f58a7f1-bc86-472f-be25-c454eead747f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:26:12.126143 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f58a7f1-bc86-472f-be25-c454eead747f service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:26:12.137254 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 696/1385] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:26:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:26:14.226413 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc5563-df4d-4a3d-8487-3a4521db33fe None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:14.227014 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc5563-df4d-4a3d-8487-3a4521db33fe None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:14.227014 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc5563-df4d-4a3d-8487-3a4521db33fe None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:14.228362 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 690/1386] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:26:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:26:14.233993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a4f53-cf1a-443f-b13b-1477391c470d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:14.234451 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a4f53-cf1a-443f-b13b-1477391c470d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:14.234567 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a4f53-cf1a-443f-b13b-1477391c470d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:14.260541 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-213a4f53-cf1a-443f-b13b-1477391c470d None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:14.262447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-213a4f53-cf1a-443f-b13b-1477391c470d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:14.274800 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 697/1387] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:26:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:26:14.320285 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc5563-df4d-4a3d-8487-3a4521db33fe None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:26:14.335236 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc5563-df4d-4a3d-8487-3a4521db33fe None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:26:14.340690 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69add73-59ee-442b-837d-871007bae12f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:26:14.341397 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69add73-59ee-442b-837d-871007bae12f service nova] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:14.341625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69add73-59ee-442b-837d-871007bae12f service nova] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:14.341904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69add73-59ee-442b-837d-871007bae12f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:14.351288 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69add73-59ee-442b-837d-871007bae12f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:26:14.352914 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69add73-59ee-442b-837d-871007bae12f service nova] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:26:14.362847 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 691/1388] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:26:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:26:16.915190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a59f0-9d06-4965-ad8b-aff43253cc61 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:16.915787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a59f0-9d06-4965-ad8b-aff43253cc61 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:16.915787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a59f0-9d06-4965-ad8b-aff43253cc61 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:16.917197 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 698/1389] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:26:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:26:16.926815 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486c74b-ff6d-4e6f-b07c-75384871d2a9 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:16.931568 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486c74b-ff6d-4e6f-b07c-75384871d2a9 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:16.931568 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486c74b-ff6d-4e6f-b07c-75384871d2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:16.946317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-9486c74b-ff6d-4e6f-b07c-75384871d2a9 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:16.947564 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-9486c74b-ff6d-4e6f-b07c-75384871d2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:16.956259 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 692/1390] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:26:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:26:17.002325 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a59f0-9d06-4965-ad8b-aff43253cc61 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:26:17.021790 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a59f0-9d06-4965-ad8b-aff43253cc61 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:26:17.027814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73e40b-7b0d-482c-b90d-3a2ca248277a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:26:17.028900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73e40b-7b0d-482c-b90d-3a2ca248277a service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:17.028900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73e40b-7b0d-482c-b90d-3a2ca248277a service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:17.029091 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73e40b-7b0d-482c-b90d-3a2ca248277a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:17.038417 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73e40b-7b0d-482c-b90d-3a2ca248277a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:26:17.040067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73e40b-7b0d-482c-b90d-3a2ca248277a service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:26:17.050163 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 699/1391] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:26:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:26:19.278980 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7a2eb-e273-441b-97d0-63835804fc2d None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:19.279669 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7a2eb-e273-441b-97d0-63835804fc2d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:19.279669 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f7a2eb-e273-441b-97d0-63835804fc2d None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:19.280936 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 693/1392] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:26:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:26:19.287691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea790c29-95da-49dd-8ff2-d46370727b91 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:19.288217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea790c29-95da-49dd-8ff2-d46370727b91 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:19.288217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea790c29-95da-49dd-8ff2-d46370727b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:19.307119 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ea790c29-95da-49dd-8ff2-d46370727b91 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:19.308382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ea790c29-95da-49dd-8ff2-d46370727b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:19.317698 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 700/1393] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:26:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:26:19.364555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7a2eb-e273-441b-97d0-63835804fc2d None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:26:19.381544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f7a2eb-e273-441b-97d0-63835804fc2d None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:26:19.387190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9a3732-ebe4-4c69-9781-d512763a50c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:26:19.387862 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a3732-ebe4-4c69-9781-d512763a50c8 service nova] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:19.388156 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a3732-ebe4-4c69-9781-d512763a50c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:19.388514 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9a3732-ebe4-4c69-9781-d512763a50c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:19.399297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a3732-ebe4-4c69-9781-d512763a50c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:26:19.401589 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9a3732-ebe4-4c69-9781-d512763a50c8 service nova] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:26:19.414947 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 694/1394] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:26:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:26:21.655889 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec3f60-c3e0-482b-a018-0cbb40dc94b9 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:21.656450 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec3f60-c3e0-482b-a018-0cbb40dc94b9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:21.656450 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ec3f60-c3e0-482b-a018-0cbb40dc94b9 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:21.657817 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 701/1395] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:26:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:26:21.663697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab8154-48d9-49e2-976e-00e8da9c7241 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:21.664120 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab8154-48d9-49e2-976e-00e8da9c7241 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:21.664291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab8154-48d9-49e2-976e-00e8da9c7241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:21.683339 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-6dab8154-48d9-49e2-976e-00e8da9c7241 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:21.684780 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-6dab8154-48d9-49e2-976e-00e8da9c7241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:21.692679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 695/1396] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:26:21.736807 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec3f60-c3e0-482b-a018-0cbb40dc94b9 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:26:21.751686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ec3f60-c3e0-482b-a018-0cbb40dc94b9 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:26:21.757582 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb89c4-d131-440f-b97d-e29a661b4b0d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:26:21.758179 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb89c4-d131-440f-b97d-e29a661b4b0d service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:21.758469 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb89c4-d131-440f-b97d-e29a661b4b0d service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:21.758753 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb89c4-d131-440f-b97d-e29a661b4b0d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:21.767295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb89c4-d131-440f-b97d-e29a661b4b0d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:26:21.769048 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb89c4-d131-440f-b97d-e29a661b4b0d service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:26:21.779418 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 702/1397] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:26:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:26:21.783297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ce7dc-76ec-48d2-84e5-79de582471bf None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:21.783848 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ce7dc-76ec-48d2-84e5-79de582471bf None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:21.784133 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75ce7dc-76ec-48d2-84e5-79de582471bf None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:21.786432 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 696/1398] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:26:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:26:21.791385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1dac5-7a0b-4170-94cb-996084682aae None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:21.791681 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1dac5-7a0b-4170-94cb-996084682aae None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:21.791961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1dac5-7a0b-4170-94cb-996084682aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:21.810039 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-45a1dac5-7a0b-4170-94cb-996084682aae None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:21.811233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-45a1dac5-7a0b-4170-94cb-996084682aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:21.819009 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 703/1399] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:26:21.876070 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dcce7-ab0f-4342-b195-72a3c8ba8216 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:21.878585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dcce7-ab0f-4342-b195-72a3c8ba8216 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:21.878585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593dcce7-ab0f-4342-b195-72a3c8ba8216 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:21.878585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 697/1400] 10.222.0.47 () {54 vars in 962 bytes} [Thu Jun 30 10:26:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:26:21.884218 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323b22f-5999-4dbd-bf1b-4470c4d1afe1 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:21.884729 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323b22f-5999-4dbd-bf1b-4470c4d1afe1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:21.885168 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8323b22f-5999-4dbd-bf1b-4470c4d1afe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:21.934135 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8323b22f-5999-4dbd-bf1b-4470c4d1afe1 None None] MFA Rules not processed for user `c7a61d85908d42698b27b9fc00daa832`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:21.935402 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8323b22f-5999-4dbd-bf1b-4470c4d1afe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:21.967169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:26:21.997596 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 704/1401] 10.222.0.47 () {58 vars in 1059 bytes} [Thu Jun 30 10:26:21 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:26:22.006235 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593dcce7-ab0f-4342-b195-72a3c8ba8216 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:26:22.021153 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593dcce7-ab0f-4342-b195-72a3c8ba8216 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:26:22.024808 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6149a-5456-472d-a53c-54205bb34f32 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7a61d85908d42698b27b9fc00daa832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:26:22.025532 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6149a-5456-472d-a53c-54205bb34f32 service glance] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:22.025909 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6149a-5456-472d-a53c-54205bb34f32 service glance] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:22.026258 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6149a-5456-472d-a53c-54205bb34f32 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:22.037812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6149a-5456-472d-a53c-54205bb34f32 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:26:22.039489 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6149a-5456-472d-a53c-54205bb34f32 service glance] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:26:22.051180 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 698/1402] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:26:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:26:23.028096 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734baf50-e111-42c9-85dc-5d36e5e6b486 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:23.028611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734baf50-e111-42c9-85dc-5d36e5e6b486 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:23.028773 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734baf50-e111-42c9-85dc-5d36e5e6b486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:23.077640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-734baf50-e111-42c9-85dc-5d36e5e6b486 None None] MFA Rules not processed for user `66ad8defb9b6453c96f1a8a7343695ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:23.078847 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-734baf50-e111-42c9-85dc-5d36e5e6b486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:23.146649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 705/1403] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jun 30 10:26:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:26:23.190886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf9a93-0f3d-4b06-ba18-43d6ee167c59 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:23.191488 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf9a93-0f3d-4b06-ba18-43d6ee167c59 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:23.191736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf9a93-0f3d-4b06-ba18-43d6ee167c59 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:23.193191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 699/1404] 10.222.0.47 () {54 vars in 942 bytes} [Thu Jun 30 10:26:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:26:23.199545 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed615d-b7ad-4126-b2ab-3ada78b49e1a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:23.199985 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed615d-b7ad-4126-b2ab-3ada78b49e1a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:23.200368 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed615d-b7ad-4126-b2ab-3ada78b49e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:23.248491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-68ed615d-b7ad-4126-b2ab-3ada78b49e1a None None] MFA Rules not processed for user `07de926cb8594b098f79111dab8a9cc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:23.249805 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-68ed615d-b7ad-4126-b2ab-3ada78b49e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:23.322332 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 706/1405] 10.222.0.47 () {58 vars in 1039 bytes} [Thu Jun 30 10:26:23 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:26:23.332064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf9a93-0f3d-4b06-ba18-43d6ee167c59 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:26:23.339618 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=61 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:26:23.345944 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf9a93-0f3d-4b06-ba18-43d6ee167c59 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:26:23.351743 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9a0a9d-b07c-489e-b8a2-24f4adca23d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07de926cb8594b098f79111dab8a9cc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:26:23.352380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a0a9d-b07c-489e-b8a2-24f4adca23d4 service swift] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:23.352666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a0a9d-b07c-489e-b8a2-24f4adca23d4 service swift] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:23.352944 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9a0a9d-b07c-489e-b8a2-24f4adca23d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:23.360910 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a0a9d-b07c-489e-b8a2-24f4adca23d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:26:23.362488 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9a0a9d-b07c-489e-b8a2-24f4adca23d4 service swift] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:26:23.370724 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 700/1406] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jun 30 10:26:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:26:24.035018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11f8fd-38d0-4120-844e-2465540c7959 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:24.035585 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11f8fd-38d0-4120-844e-2465540c7959 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:24.035669 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf11f8fd-38d0-4120-844e-2465540c7959 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:24.038381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 707/1407] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:26:24.042791 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548244f-5b72-4f55-812a-9d4fc27d8474 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:24.043083 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548244f-5b72-4f55-812a-9d4fc27d8474 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:24.043400 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7548244f-5b72-4f55-812a-9d4fc27d8474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:24.061935 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-7548244f-5b72-4f55-812a-9d4fc27d8474 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:24.063141 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-7548244f-5b72-4f55-812a-9d4fc27d8474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:24.071885 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 701/1408] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:26:24.118342 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11f8fd-38d0-4120-844e-2465540c7959 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:26:24.134339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf11f8fd-38d0-4120-844e-2465540c7959 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:26:24.140149 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613ec89-9b7d-4f91-863d-d0348ff4bad5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:26:24.140880 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613ec89-9b7d-4f91-863d-d0348ff4bad5 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:24.141227 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613ec89-9b7d-4f91-863d-d0348ff4bad5 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:24.141593 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613ec89-9b7d-4f91-863d-d0348ff4bad5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:24.151863 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613ec89-9b7d-4f91-863d-d0348ff4bad5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:26:24.153146 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613ec89-9b7d-4f91-863d-d0348ff4bad5 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:26:24.158525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:26:24.163658 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 708/1409] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:26:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:26:34.056675 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=8 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:26:47.731142 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387960d6-54e6-4ee0-91d7-26051f44f273 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:47.732047 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387960d6-54e6-4ee0-91d7-26051f44f273 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:47.732047 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387960d6-54e6-4ee0-91d7-26051f44f273 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:47.733128 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 702/1410] 10.222.0.47 () {54 vars in 918 bytes} [Thu Jun 30 10:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:26:47.738664 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178dad5-03eb-4dda-85c7-354b73a905da None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:47.739180 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178dad5-03eb-4dda-85c7-354b73a905da None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:47.739598 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178dad5-03eb-4dda-85c7-354b73a905da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:47.759028 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9178dad5-03eb-4dda-85c7-354b73a905da None None] MFA Rules not processed for user `46c99f43a9344d7d92632990e45bb9cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:47.760187 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9178dad5-03eb-4dda-85c7-354b73a905da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:47.773101 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 709/1411] 10.222.0.47 () {58 vars in 1015 bytes} [Thu Jun 30 10:26:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:26:47.789033 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffead55d-ba01-46f7-8fea-801e529ca0b1 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:47.789319 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffead55d-ba01-46f7-8fea-801e529ca0b1 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:47.789595 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffead55d-ba01-46f7-8fea-801e529ca0b1 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:47.790965 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 703/1412] 10.222.0.47 () {54 vars in 918 bytes} [Thu Jun 30 10:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:26:47.796634 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df462e45-2156-48b5-afb1-d25f22329337 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:47.796923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df462e45-2156-48b5-afb1-d25f22329337 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:47.797208 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df462e45-2156-48b5-afb1-d25f22329337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:47.820213 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-df462e45-2156-48b5-afb1-d25f22329337 None None] MFA Rules not processed for user `71f6d7715e6f49ffb9fdf6e1307bbd51`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:47.821563 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-df462e45-2156-48b5-afb1-d25f22329337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:47.832856 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 710/1413] 10.222.0.47 () {58 vars in 1015 bytes} [Thu Jun 30 10:26:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:26:50.381383 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a5b9a-02d0-4382-9f77-bbeaaaddf4ce None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:50.382092 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a5b9a-02d0-4382-9f77-bbeaaaddf4ce None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:50.382092 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990a5b9a-02d0-4382-9f77-bbeaaaddf4ce None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:50.383211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 10:26:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:26:50.388517 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30a975-d8c4-4143-b786-4fe674650ae3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:50.388977 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30a975-d8c4-4143-b786-4fe674650ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:50.389108 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf30a975-d8c4-4143-b786-4fe674650ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:50.407187 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-cf30a975-d8c4-4143-b786-4fe674650ae3 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:50.408456 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cf30a975-d8c4-4143-b786-4fe674650ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:50.416693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 10:26:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:26:50.461262 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a5b9a-02d0-4382-9f77-bbeaaaddf4ce None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:26:50.475243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990a5b9a-02d0-4382-9f77-bbeaaaddf4ce None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:26:50.480704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b10d0e-2c0d-459f-9439-913ce021e890 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:26:50.481338 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b10d0e-2c0d-459f-9439-913ce021e890 service nova] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:50.481622 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b10d0e-2c0d-459f-9439-913ce021e890 service nova] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:50.481900 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b10d0e-2c0d-459f-9439-913ce021e890 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:50.490933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b10d0e-2c0d-459f-9439-913ce021e890 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:26:50.492558 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b10d0e-2c0d-459f-9439-913ce021e890 service nova] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:26:50.502116 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 705/1416] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:26:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:26:51.397397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcd8ac-4ed9-44eb-86d7-03a5b6a49d1c None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:51.398000 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcd8ac-4ed9-44eb-86d7-03a5b6a49d1c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:51.398000 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcd8ac-4ed9-44eb-86d7-03a5b6a49d1c None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:51.399449 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 10:26:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:26:51.404857 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a61df-4360-424f-8177-6e4194e02369 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:51.405296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a61df-4360-424f-8177-6e4194e02369 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:51.405423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a61df-4360-424f-8177-6e4194e02369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:51.423998 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-e61a61df-4360-424f-8177-6e4194e02369 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:51.425349 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-e61a61df-4360-424f-8177-6e4194e02369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:51.433283 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 10:26:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:26:51.598504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adcd8ac-4ed9-44eb-86d7-03a5b6a49d1c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:26:51.612870 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adcd8ac-4ed9-44eb-86d7-03a5b6a49d1c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:26:51.618327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52872030-39a1-4f1d-8573-b5331f1aeccd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:26:51.618992 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52872030-39a1-4f1d-8573-b5331f1aeccd service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:51.619309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52872030-39a1-4f1d-8573-b5331f1aeccd service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:51.619603 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52872030-39a1-4f1d-8573-b5331f1aeccd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:51.628628 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52872030-39a1-4f1d-8573-b5331f1aeccd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:26:51.630333 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52872030-39a1-4f1d-8573-b5331f1aeccd service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:26:51.639705 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 713/1419] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:26:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:26:52.531081 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a7a72-19e7-4234-b218-5c169992c356 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:52.531583 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a7a72-19e7-4234-b218-5c169992c356 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:52.531689 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801a7a72-19e7-4234-b218-5c169992c356 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:52.533098 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 10:26:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:26:52.538679 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069aa57-6ee1-426c-93a4-eccdc0d37000 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:52.539142 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069aa57-6ee1-426c-93a4-eccdc0d37000 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:52.539244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c069aa57-6ee1-426c-93a4-eccdc0d37000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:52.557017 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c069aa57-6ee1-426c-93a4-eccdc0d37000 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:52.558213 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c069aa57-6ee1-426c-93a4-eccdc0d37000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:52.566246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 10:26:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:26:52.716557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a7a72-19e7-4234-b218-5c169992c356 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:26:52.731495 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801a7a72-19e7-4234-b218-5c169992c356 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:26:52.737007 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df3eae-1f7f-45f4-9af5-608abb95f29c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:26:52.787522 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df3eae-1f7f-45f4-9af5-608abb95f29c service nova] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:52.787522 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df3eae-1f7f-45f4-9af5-608abb95f29c service nova] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:52.787522 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df3eae-1f7f-45f4-9af5-608abb95f29c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:52.787522 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df3eae-1f7f-45f4-9af5-608abb95f29c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:26:52.787522 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df3eae-1f7f-45f4-9af5-608abb95f29c service nova] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:26:52.787522 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 708/1422] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:26:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:26:53.639844 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd1546-a3d8-4bc8-8f4a-6e45c91bb8af None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:53.640412 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd1546-a3d8-4bc8-8f4a-6e45c91bb8af None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:53.640412 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cd1546-a3d8-4bc8-8f4a-6e45c91bb8af None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:53.641794 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 10:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:26:53.646615 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241f55f-2cc3-43d4-8ca6-fc07a8ecdd28 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:53.647031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241f55f-2cc3-43d4-8ca6-fc07a8ecdd28 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:53.647178 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241f55f-2cc3-43d4-8ca6-fc07a8ecdd28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:53.665480 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-8241f55f-2cc3-43d4-8ca6-fc07a8ecdd28 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:53.666673 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-8241f55f-2cc3-43d4-8ca6-fc07a8ecdd28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:53.674470 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 10:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:26:53.825829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd1546-a3d8-4bc8-8f4a-6e45c91bb8af None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:26:53.839998 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cd1546-a3d8-4bc8-8f4a-6e45c91bb8af None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:26:53.845355 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0276d03-71b5-48e3-a95e-55224fcad999 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:26:53.845984 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0276d03-71b5-48e3-a95e-55224fcad999 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:53.846270 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0276d03-71b5-48e3-a95e-55224fcad999 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:53.847151 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0276d03-71b5-48e3-a95e-55224fcad999 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:53.855288 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0276d03-71b5-48e3-a95e-55224fcad999 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:26:53.856897 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0276d03-71b5-48e3-a95e-55224fcad999 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:26:53.866870 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 716/1425] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:26:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:26:54.730685 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22bb05-ae83-4ae0-ba52-8c7e992b5799 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:54.731189 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22bb05-ae83-4ae0-ba52-8c7e992b5799 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:54.731332 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d22bb05-ae83-4ae0-ba52-8c7e992b5799 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:54.732637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 10:26:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:26:54.737693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6346cb-5d38-4a45-945c-9d23b1002fbf None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:54.738098 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6346cb-5d38-4a45-945c-9d23b1002fbf None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:54.738272 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6346cb-5d38-4a45-945c-9d23b1002fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:54.757543 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ec6346cb-5d38-4a45-945c-9d23b1002fbf None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:54.758668 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ec6346cb-5d38-4a45-945c-9d23b1002fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:54.766528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 10:26:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:26:54.919825 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d22bb05-ae83-4ae0-ba52-8c7e992b5799 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:26:54.934544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d22bb05-ae83-4ae0-ba52-8c7e992b5799 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:26:54.939781 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e700e-2d4a-40c7-aad4-bb5608483156 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:26:54.941160 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e700e-2d4a-40c7-aad4-bb5608483156 service nova] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:54.941462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e700e-2d4a-40c7-aad4-bb5608483156 service nova] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:54.941741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e700e-2d4a-40c7-aad4-bb5608483156 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:54.949869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e700e-2d4a-40c7-aad4-bb5608483156 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:26:54.951528 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e700e-2d4a-40c7-aad4-bb5608483156 service nova] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:26:54.960877 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 711/1428] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:26:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:26:55.862781 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555835ed-1c6d-43f3-af54-78c19d3efd3e None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:55.863400 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555835ed-1c6d-43f3-af54-78c19d3efd3e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:55.863400 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555835ed-1c6d-43f3-af54-78c19d3efd3e None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:55.864658 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 10:26:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:26:55.869447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab88c4-f2f9-4240-9030-b5c923e8e8de None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:55.869847 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab88c4-f2f9-4240-9030-b5c923e8e8de None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:55.870004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ab88c4-f2f9-4240-9030-b5c923e8e8de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:55.887879 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-99ab88c4-f2f9-4240-9030-b5c923e8e8de None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:55.889061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-99ab88c4-f2f9-4240-9030-b5c923e8e8de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:55.897613 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 10:26:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:26:56.049533 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555835ed-1c6d-43f3-af54-78c19d3efd3e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:26:56.064712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555835ed-1c6d-43f3-af54-78c19d3efd3e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:26:56.070954 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32728da2-9d8c-4c20-8956-7168098fcae1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:26:56.071662 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32728da2-9d8c-4c20-8956-7168098fcae1 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:56.072043 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32728da2-9d8c-4c20-8956-7168098fcae1 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:56.072321 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32728da2-9d8c-4c20-8956-7168098fcae1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:56.080424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32728da2-9d8c-4c20-8956-7168098fcae1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:26:56.082735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32728da2-9d8c-4c20-8956-7168098fcae1 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:26:56.092131 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 719/1431] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:26:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:26:57.000298 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0cece-5702-40a6-baae-7abc359ca207 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:57.000848 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0cece-5702-40a6-baae-7abc359ca207 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:57.000848 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0cece-5702-40a6-baae-7abc359ca207 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:57.002216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 10:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:26:57.007231 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2170a8-f220-496e-bb22-91516b06447a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:57.007683 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2170a8-f220-496e-bb22-91516b06447a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:57.007833 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2170a8-f220-496e-bb22-91516b06447a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:57.026105 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5e2170a8-f220-496e-bb22-91516b06447a None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:57.027578 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5e2170a8-f220-496e-bb22-91516b06447a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:57.035954 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 10:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:26:57.199801 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0cece-5702-40a6-baae-7abc359ca207 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:26:57.214740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0cece-5702-40a6-baae-7abc359ca207 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:26:57.220005 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484e85f6-4084-4ed6-9b44-30ae8e175679 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:26:57.220644 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e85f6-4084-4ed6-9b44-30ae8e175679 service nova] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:57.220959 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e85f6-4084-4ed6-9b44-30ae8e175679 service nova] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:57.221238 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484e85f6-4084-4ed6-9b44-30ae8e175679 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:57.229949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e85f6-4084-4ed6-9b44-30ae8e175679 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:26:57.231558 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484e85f6-4084-4ed6-9b44-30ae8e175679 service nova] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:26:57.240768 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 714/1434] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:26:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:26:58.150492 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c15b5e-7012-417b-b53f-1142ae026940 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:58.151064 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c15b5e-7012-417b-b53f-1142ae026940 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:58.151064 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c15b5e-7012-417b-b53f-1142ae026940 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:58.152622 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 10:26:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:26:58.157565 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfc6a2-30c2-429c-bccc-99bc916a0c47 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:58.158006 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfc6a2-30c2-429c-bccc-99bc916a0c47 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:58.158122 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cfc6a2-30c2-429c-bccc-99bc916a0c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:58.176872 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-00cfc6a2-30c2-429c-bccc-99bc916a0c47 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:58.178134 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-00cfc6a2-30c2-429c-bccc-99bc916a0c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:58.185931 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 10:26:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:26:58.336471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c15b5e-7012-417b-b53f-1142ae026940 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:26:58.351201 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c15b5e-7012-417b-b53f-1142ae026940 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:26:58.356460 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4e871-1734-4271-b3d8-772d592c4b16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:26:58.357087 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4e871-1734-4271-b3d8-772d592c4b16 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:58.357371 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4e871-1734-4271-b3d8-772d592c4b16 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:58.357645 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4e871-1734-4271-b3d8-772d592c4b16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:58.366387 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4e871-1734-4271-b3d8-772d592c4b16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:26:58.368029 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4e871-1734-4271-b3d8-772d592c4b16 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:26:58.377468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 722/1437] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:26:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:26:59.283462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb74f4-75f9-4d2f-8f19-c84aa53c1e89 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:59.284035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb74f4-75f9-4d2f-8f19-c84aa53c1e89 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:59.284260 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfb74f4-75f9-4d2f-8f19-c84aa53c1e89 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:59.285896 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 10:26:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:26:59.291410 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e79047-0631-4351-91ba-5ef2813367e6 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:59.291838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e79047-0631-4351-91ba-5ef2813367e6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:59.291972 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e79047-0631-4351-91ba-5ef2813367e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:59.310357 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-51e79047-0631-4351-91ba-5ef2813367e6 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:26:59.311587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-51e79047-0631-4351-91ba-5ef2813367e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:26:59.320685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 10:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:26:59.474550 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb74f4-75f9-4d2f-8f19-c84aa53c1e89 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:26:59.489933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfb74f4-75f9-4d2f-8f19-c84aa53c1e89 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:26:59.495246 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8a50fa-63d1-49e9-9d13-9aa8c6ebb008 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:26:59.495914 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a50fa-63d1-49e9-9d13-9aa8c6ebb008 service nova] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:26:59.496200 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a50fa-63d1-49e9-9d13-9aa8c6ebb008 service nova] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:26:59.496479 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a50fa-63d1-49e9-9d13-9aa8c6ebb008 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:26:59.505893 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a50fa-63d1-49e9-9d13-9aa8c6ebb008 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:26:59.507531 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8a50fa-63d1-49e9-9d13-9aa8c6ebb008 service nova] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:26:59.517533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 717/1440] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:26:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:27:01.490368 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e0d49-b9ac-4d23-9c94-6953040ca74c None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:01.490921 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e0d49-b9ac-4d23-9c94-6953040ca74c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:01.490921 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15e0d49-b9ac-4d23-9c94-6953040ca74c None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:01.492362 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 10:27:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:27:01.497759 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c7ff7-c876-40e3-a5c0-17d6ed6e21a8 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:01.498175 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c7ff7-c876-40e3-a5c0-17d6ed6e21a8 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:01.498355 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55c7ff7-c876-40e3-a5c0-17d6ed6e21a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:01.517297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-f55c7ff7-c876-40e3-a5c0-17d6ed6e21a8 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:27:01.518517 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-f55c7ff7-c876-40e3-a5c0-17d6ed6e21a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:27:01.526534 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 10:27:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:27:01.569450 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e0d49-b9ac-4d23-9c94-6953040ca74c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:27:01.585015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15e0d49-b9ac-4d23-9c94-6953040ca74c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:27:01.590347 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1546e20-020a-4c75-9bf2-17683ec577a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:27:01.591380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1546e20-020a-4c75-9bf2-17683ec577a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:01.591814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1546e20-020a-4c75-9bf2-17683ec577a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:01.592235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1546e20-020a-4c75-9bf2-17683ec577a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:01.603739 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1546e20-020a-4c75-9bf2-17683ec577a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:27:01.605187 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1546e20-020a-4c75-9bf2-17683ec577a3 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:27:01.615255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 725/1443] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:27:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:27:03.836034 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b7041-bed5-4ff5-a2ba-c7f76d2acf8c None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:03.836567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b7041-bed5-4ff5-a2ba-c7f76d2acf8c None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:03.836567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b7041-bed5-4ff5-a2ba-c7f76d2acf8c None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:03.837866 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 10:27:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:27:03.843141 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d003b7-3a8e-458e-a66f-8efe039a5b22 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:03.843589 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d003b7-3a8e-458e-a66f-8efe039a5b22 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:03.843759 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d003b7-3a8e-458e-a66f-8efe039a5b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:03.861392 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-25d003b7-3a8e-458e-a66f-8efe039a5b22 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:27:03.862582 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-25d003b7-3a8e-458e-a66f-8efe039a5b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:27:03.871020 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 10:27:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:27:03.909042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b7041-bed5-4ff5-a2ba-c7f76d2acf8c None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:27:03.919851 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b7041-bed5-4ff5-a2ba-c7f76d2acf8c None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:27:03.923377 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65bf9ea-b107-4b9d-8f55-5051a62c0c33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:27:03.924014 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bf9ea-b107-4b9d-8f55-5051a62c0c33 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:03.924301 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bf9ea-b107-4b9d-8f55-5051a62c0c33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:03.924581 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65bf9ea-b107-4b9d-8f55-5051a62c0c33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:03.933913 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bf9ea-b107-4b9d-8f55-5051a62c0c33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:27:03.935372 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65bf9ea-b107-4b9d-8f55-5051a62c0c33 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:27:03.945555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 720/1446] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:27:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:27:06.358170 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591aeea-cf93-426d-a65d-33691aefdd9b None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:06.358768 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591aeea-cf93-426d-a65d-33691aefdd9b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:06.358768 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9591aeea-cf93-426d-a65d-33691aefdd9b None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:06.360264 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 10:27:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:27:06.366077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c945fd8-eca0-4278-aeae-e79a619f7bee None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:06.366547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c945fd8-eca0-4278-aeae-e79a619f7bee None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:06.366643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c945fd8-eca0-4278-aeae-e79a619f7bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:06.384927 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-4c945fd8-eca0-4278-aeae-e79a619f7bee None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:27:06.386114 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-4c945fd8-eca0-4278-aeae-e79a619f7bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:27:06.393703 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 10:27:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:27:06.433113 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591aeea-cf93-426d-a65d-33691aefdd9b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:27:06.443593 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9591aeea-cf93-426d-a65d-33691aefdd9b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:27:06.447226 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec1709-38d1-4542-b60c-42c26abb6303 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:27:06.447907 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec1709-38d1-4542-b60c-42c26abb6303 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:06.448190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec1709-38d1-4542-b60c-42c26abb6303 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:06.448467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec1709-38d1-4542-b60c-42c26abb6303 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:06.457373 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec1709-38d1-4542-b60c-42c26abb6303 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:27:06.458802 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec1709-38d1-4542-b60c-42c26abb6303 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:27:06.468217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 728/1449] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:27:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:27:08.667986 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82a64e-b765-45ac-94c4-284a1a25e0ff None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:08.668550 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82a64e-b765-45ac-94c4-284a1a25e0ff None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:08.668550 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e82a64e-b765-45ac-94c4-284a1a25e0ff None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:08.670036 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 10:27:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:27:08.674961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63665d7c-9161-44f5-9640-dfa544af6ed3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:08.675450 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63665d7c-9161-44f5-9640-dfa544af6ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:08.675571 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63665d7c-9161-44f5-9640-dfa544af6ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:08.693856 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-63665d7c-9161-44f5-9640-dfa544af6ed3 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:27:08.695064 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-63665d7c-9161-44f5-9640-dfa544af6ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:27:08.702989 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 10:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:27:08.742140 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e82a64e-b765-45ac-94c4-284a1a25e0ff None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:27:08.752650 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e82a64e-b765-45ac-94c4-284a1a25e0ff None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:27:08.756817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63f8c3-25c4-46b9-9b89-b4789bb912a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:27:08.757392 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63f8c3-25c4-46b9-9b89-b4789bb912a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:08.757676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63f8c3-25c4-46b9-9b89-b4789bb912a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:08.757953 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63f8c3-25c4-46b9-9b89-b4789bb912a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:08.766825 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce63f8c3-25c4-46b9-9b89-b4789bb912a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:27:08.768277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce63f8c3-25c4-46b9-9b89-b4789bb912a1 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:27:08.778491 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 723/1452] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:27:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:27:10.940239 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc3e39-3b11-4967-902f-d8e688faf136 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:10.940792 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc3e39-3b11-4967-902f-d8e688faf136 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:10.941418 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bc3e39-3b11-4967-902f-d8e688faf136 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:10.942822 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 10:27:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:27:10.948314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c549d-c825-4ff8-9f84-de328d1dfde4 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:10.948745 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c549d-c825-4ff8-9f84-de328d1dfde4 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:10.948877 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63c549d-c825-4ff8-9f84-de328d1dfde4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:10.966776 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-f63c549d-c825-4ff8-9f84-de328d1dfde4 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:27:10.968008 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-f63c549d-c825-4ff8-9f84-de328d1dfde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:27:10.975947 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 10:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:27:11.013492 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc3e39-3b11-4967-902f-d8e688faf136 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:27:11.024828 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bc3e39-3b11-4967-902f-d8e688faf136 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:27:11.028659 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c9bc25-8e2a-4a21-b0a8-25e00dc82126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:27:11.029294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9bc25-8e2a-4a21-b0a8-25e00dc82126 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:11.029579 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9bc25-8e2a-4a21-b0a8-25e00dc82126 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:11.029857 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c9bc25-8e2a-4a21-b0a8-25e00dc82126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:11.038874 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9bc25-8e2a-4a21-b0a8-25e00dc82126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:27:11.040348 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c9bc25-8e2a-4a21-b0a8-25e00dc82126 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:27:11.049752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 731/1455] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:27:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:27:13.145094 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a456b98-e22b-4698-b6a0-3e48f74d4da5 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:13.145675 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a456b98-e22b-4698-b6a0-3e48f74d4da5 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:13.145675 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a456b98-e22b-4698-b6a0-3e48f74d4da5 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:13.147318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 10:27:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:27:13.152692 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a8df7-89c5-4092-9fc7-be09757d59a2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:13.153096 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a8df7-89c5-4092-9fc7-be09757d59a2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:13.153260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8a8df7-89c5-4092-9fc7-be09757d59a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:13.171432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-2c8a8df7-89c5-4092-9fc7-be09757d59a2 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:27:13.172639 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2c8a8df7-89c5-4092-9fc7-be09757d59a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:27:13.181309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 10:27:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:27:13.218535 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a456b98-e22b-4698-b6a0-3e48f74d4da5 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:27:13.230030 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a456b98-e22b-4698-b6a0-3e48f74d4da5 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:27:13.233446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9344ba10-28ab-456b-a266-f63068f9bf8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:27:13.234075 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344ba10-28ab-456b-a266-f63068f9bf8f service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:13.234354 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344ba10-28ab-456b-a266-f63068f9bf8f service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:13.234625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9344ba10-28ab-456b-a266-f63068f9bf8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:13.243923 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9344ba10-28ab-456b-a266-f63068f9bf8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:27:13.245330 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9344ba10-28ab-456b-a266-f63068f9bf8f service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:27:13.254898 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 726/1458] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:27:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:27:15.356530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664bdc73-c09a-450f-8f77-d8146542ce0a None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:15.357078 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664bdc73-c09a-450f-8f77-d8146542ce0a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:15.357202 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664bdc73-c09a-450f-8f77-d8146542ce0a None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:15.358531 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 10:27:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:27:15.364309 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775745dc-65ad-417c-92b5-8505bcaf1ad6 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:15.364714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775745dc-65ad-417c-92b5-8505bcaf1ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:15.364861 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775745dc-65ad-417c-92b5-8505bcaf1ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:15.370829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=70 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:27:15.383150 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-775745dc-65ad-417c-92b5-8505bcaf1ad6 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:27:15.384373 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-775745dc-65ad-417c-92b5-8505bcaf1ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:27:15.392256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 10:27:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:27:15.430608 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664bdc73-c09a-450f-8f77-d8146542ce0a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:27:15.435996 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=73 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:27:15.441561 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664bdc73-c09a-450f-8f77-d8146542ce0a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:27:15.445065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf569f5-4d71-4500-9082-0fee4ee4cc3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:27:15.450885 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf569f5-4d71-4500-9082-0fee4ee4cc3a service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:15.450885 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf569f5-4d71-4500-9082-0fee4ee4cc3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:15.450885 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf569f5-4d71-4500-9082-0fee4ee4cc3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:15.455175 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf569f5-4d71-4500-9082-0fee4ee4cc3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:27:15.456682 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf569f5-4d71-4500-9082-0fee4ee4cc3a service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:27:15.466007 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 734/1461] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:27:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:27:17.566710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a5a84-1176-4706-8a73-aa894b6f6a9d None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:17.567236 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a5a84-1176-4706-8a73-aa894b6f6a9d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:17.567363 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a5a84-1176-4706-8a73-aa894b6f6a9d None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:17.568718 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 10:27:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:27:17.574120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565428e4-ae95-40c3-85e9-7824201c9257 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:17.574554 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565428e4-ae95-40c3-85e9-7824201c9257 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:17.574677 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565428e4-ae95-40c3-85e9-7824201c9257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:17.594697 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-565428e4-ae95-40c3-85e9-7824201c9257 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:27:17.595947 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-565428e4-ae95-40c3-85e9-7824201c9257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:27:17.604260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 10:27:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:27:17.648973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a5a84-1176-4706-8a73-aa894b6f6a9d None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:27:17.660757 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a5a84-1176-4706-8a73-aa894b6f6a9d None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:27:17.664185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f6d876-1842-4c6d-9c54-59f6ca366dd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:27:17.664817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6d876-1842-4c6d-9c54-59f6ca366dd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:17.665099 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6d876-1842-4c6d-9c54-59f6ca366dd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:17.665373 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f6d876-1842-4c6d-9c54-59f6ca366dd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:17.674392 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6d876-1842-4c6d-9c54-59f6ca366dd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:27:17.676743 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f6d876-1842-4c6d-9c54-59f6ca366dd8 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:27:17.685970 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 729/1464] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:27:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:27:19.751963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c617fe-d1d3-45e4-9b90-037df6b639e6 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:19.752501 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c617fe-d1d3-45e4-9b90-037df6b639e6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:19.752501 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c617fe-d1d3-45e4-9b90-037df6b639e6 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:19.754013 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 10:27:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:27:19.759283 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c5fb0-6f5f-42b4-8bdd-a657ef28791e None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:19.759712 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c5fb0-6f5f-42b4-8bdd-a657ef28791e None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:19.759877 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c5fb0-6f5f-42b4-8bdd-a657ef28791e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:19.778241 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-4e2c5fb0-6f5f-42b4-8bdd-a657ef28791e None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:27:19.779458 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-4e2c5fb0-6f5f-42b4-8bdd-a657ef28791e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:27:19.788080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 10:27:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:27:19.824357 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c617fe-d1d3-45e4-9b90-037df6b639e6 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:27:19.835448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c617fe-d1d3-45e4-9b90-037df6b639e6 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:27:19.838907 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67006354-6a82-4fd3-9c2a-ffaebdfc51e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:27:19.839570 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67006354-6a82-4fd3-9c2a-ffaebdfc51e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:19.839857 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67006354-6a82-4fd3-9c2a-ffaebdfc51e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:19.840135 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67006354-6a82-4fd3-9c2a-ffaebdfc51e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:19.849623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67006354-6a82-4fd3-9c2a-ffaebdfc51e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:27:19.851048 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67006354-6a82-4fd3-9c2a-ffaebdfc51e6 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:27:19.860365 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 737/1467] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:27:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:27:21.932698 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9922847-a27c-45a3-86a1-80ffcacdf9d7 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:21.933241 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9922847-a27c-45a3-86a1-80ffcacdf9d7 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:21.933241 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9922847-a27c-45a3-86a1-80ffcacdf9d7 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:21.934600 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 10:27:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:27:21.939732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6e98f-e826-45c7-af93-099a6d951859 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:21.940113 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6e98f-e826-45c7-af93-099a6d951859 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:21.940322 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c6e98f-e826-45c7-af93-099a6d951859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:21.958432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b8c6e98f-e826-45c7-af93-099a6d951859 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:27:21.959681 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b8c6e98f-e826-45c7-af93-099a6d951859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:27:21.967397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 10:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:27:22.004822 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9922847-a27c-45a3-86a1-80ffcacdf9d7 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:27:22.015434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9922847-a27c-45a3-86a1-80ffcacdf9d7 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:27:22.019006 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcbb9ce-dc0b-4213-835d-7c797c82a803 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:27:22.019673 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbb9ce-dc0b-4213-835d-7c797c82a803 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:22.019961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbb9ce-dc0b-4213-835d-7c797c82a803 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:22.020242 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcbb9ce-dc0b-4213-835d-7c797c82a803 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:22.029407 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcbb9ce-dc0b-4213-835d-7c797c82a803 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:27:22.030838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcbb9ce-dc0b-4213-835d-7c797c82a803 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:27:22.040412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 732/1470] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:27:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:27:31.953346 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=17 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:27:32.036279 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=17 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:27:42.211125 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842a3b3-9797-433b-a2e6-02f21f9fa30a None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:42.211716 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842a3b3-9797-433b-a2e6-02f21f9fa30a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:42.211716 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c842a3b3-9797-433b-a2e6-02f21f9fa30a None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:42.213099 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 739/1471] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:27:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:27:42.218437 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ac286-54c0-41bc-96ae-5ae9d54a95c6 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:42.218867 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ac286-54c0-41bc-96ae-5ae9d54a95c6 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:42.218998 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3ac286-54c0-41bc-96ae-5ae9d54a95c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:42.237235 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-ea3ac286-54c0-41bc-96ae-5ae9d54a95c6 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:27:42.238438 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-ea3ac286-54c0-41bc-96ae-5ae9d54a95c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:27:42.246173 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 733/1472] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:27:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:27:42.293946 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d7f4a-2ae1-49cb-9d8c-c21908b33b03 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:42.294235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d7f4a-2ae1-49cb-9d8c-c21908b33b03 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:42.294512 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d7f4a-2ae1-49cb-9d8c-c21908b33b03 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:42.295936 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 740/1473] 10.222.0.47 () {54 vars in 962 bytes} [Thu Jun 30 10:27:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:27:42.301181 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e6fa9-f920-49dd-8c95-d59644a4ca0a None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:42.301500 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e6fa9-f920-49dd-8c95-d59644a4ca0a None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:42.301786 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148e6fa9-f920-49dd-8c95-d59644a4ca0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:42.320403 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-148e6fa9-f920-49dd-8c95-d59644a4ca0a None None] MFA Rules not processed for user `c7a61d85908d42698b27b9fc00daa832`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:27:42.321611 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-148e6fa9-f920-49dd-8c95-d59644a4ca0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:27:42.328131 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 734/1474] 10.222.0.47 () {58 vars in 1059 bytes} [Thu Jun 30 10:27:42 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:27:42.335715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d7f4a-2ae1-49cb-9d8c-c21908b33b03 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:27:42.346835 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d7f4a-2ae1-49cb-9d8c-c21908b33b03 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:27:42.350309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a3361-a5ad-4482-b48b-3621e51855fe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7a61d85908d42698b27b9fc00daa832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:27:42.350963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a3361-a5ad-4482-b48b-3621e51855fe service glance] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:42.351247 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a3361-a5ad-4482-b48b-3621e51855fe service glance] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:42.351552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a3361-a5ad-4482-b48b-3621e51855fe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:42.360435 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a3361-a5ad-4482-b48b-3621e51855fe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:27:42.361872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a3361-a5ad-4482-b48b-3621e51855fe service glance] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:27:42.371435 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 741/1475] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:27:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:27:44.517215 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6bf7b-be4b-46f6-8e72-4930ca68a178 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:44.517676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6bf7b-be4b-46f6-8e72-4930ca68a178 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:44.517851 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6bf7b-be4b-46f6-8e72-4930ca68a178 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:44.519234 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 735/1476] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:27:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:27:44.524455 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c738634-d924-43ff-a090-2882b4fb0884 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:44.524915 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c738634-d924-43ff-a090-2882b4fb0884 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:44.525014 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c738634-d924-43ff-a090-2882b4fb0884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:44.542926 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8c738634-d924-43ff-a090-2882b4fb0884 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:27:44.544144 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8c738634-d924-43ff-a090-2882b4fb0884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:27:44.552770 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 742/1477] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:27:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:27:44.576902 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6bf7b-be4b-46f6-8e72-4930ca68a178 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:27:44.589519 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6bf7b-be4b-46f6-8e72-4930ca68a178 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:27:44.593992 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bff701-b0c0-4e4d-ab1b-a0d78a8b2052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:27:44.594638 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bff701-b0c0-4e4d-ab1b-a0d78a8b2052 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:44.594921 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bff701-b0c0-4e4d-ab1b-a0d78a8b2052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:44.595203 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bff701-b0c0-4e4d-ab1b-a0d78a8b2052 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:44.596211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bff701-b0c0-4e4d-ab1b-a0d78a8b2052 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:27:44.597980 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: WARNING py.warnings [None req-06bff701-b0c0-4e4d-ab1b-a0d78a8b2052 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 10:27:44.597980 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: warnings.warn(msg) Jun 30 10:27:44.597980 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: Jun 30 10:27:44.599136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bff701-b0c0-4e4d-ab1b-a0d78a8b2052 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:27:44.618757 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 736/1478] 10.222.0.47 () {56 vars in 1108 bytes} [Thu Jun 30 10:27:44 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:27:46.702143 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86382278-30fe-4447-92e1-7e3070463f7d None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:46.702606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86382278-30fe-4447-92e1-7e3070463f7d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:46.702733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86382278-30fe-4447-92e1-7e3070463f7d None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:46.704158 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 743/1479] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:27:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:27:46.710012 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde98749-c003-4808-84f6-1d8d9b9b38a2 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:46.710450 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde98749-c003-4808-84f6-1d8d9b9b38a2 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:46.710559 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde98749-c003-4808-84f6-1d8d9b9b38a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:46.733443 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-fde98749-c003-4808-84f6-1d8d9b9b38a2 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:27:46.734680 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-fde98749-c003-4808-84f6-1d8d9b9b38a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:27:46.744429 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 737/1480] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:27:46.790581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86382278-30fe-4447-92e1-7e3070463f7d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:27:46.803524 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86382278-30fe-4447-92e1-7e3070463f7d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:27:46.809052 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e104ac5f-861c-4159-8374-5eec7e7b1fef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:27:46.809792 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104ac5f-861c-4159-8374-5eec7e7b1fef service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:46.810129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104ac5f-861c-4159-8374-5eec7e7b1fef service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:46.810409 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e104ac5f-861c-4159-8374-5eec7e7b1fef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:46.818552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e104ac5f-861c-4159-8374-5eec7e7b1fef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:27:46.820229 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e104ac5f-861c-4159-8374-5eec7e7b1fef service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:27:46.829812 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 744/1481] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:27:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:27:49.234941 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852a4ff-f2e2-4dd1-8c32-3cdb29c1b052 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:49.235511 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852a4ff-f2e2-4dd1-8c32-3cdb29c1b052 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:49.235511 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852a4ff-f2e2-4dd1-8c32-3cdb29c1b052 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:49.236980 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 738/1482] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:27:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:27:49.243137 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05866b-571d-42e9-a707-8fb6c11d806f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:49.243679 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05866b-571d-42e9-a707-8fb6c11d806f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:49.243777 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05866b-571d-42e9-a707-8fb6c11d806f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:49.263323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0e05866b-571d-42e9-a707-8fb6c11d806f None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:27:49.264534 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0e05866b-571d-42e9-a707-8fb6c11d806f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:27:49.273221 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 745/1483] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:27:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:27:49.309002 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852a4ff-f2e2-4dd1-8c32-3cdb29c1b052 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:27:49.320154 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852a4ff-f2e2-4dd1-8c32-3cdb29c1b052 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:27:49.323696 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f6d4a3-15d5-427c-af33-d95b4d3a4871 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:27:49.324333 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6d4a3-15d5-427c-af33-d95b4d3a4871 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:49.324618 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6d4a3-15d5-427c-af33-d95b4d3a4871 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:49.324897 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f6d4a3-15d5-427c-af33-d95b4d3a4871 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:49.334900 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f6d4a3-15d5-427c-af33-d95b4d3a4871 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:27:49.336402 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f6d4a3-15d5-427c-af33-d95b4d3a4871 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:27:49.345929 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 739/1484] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:27:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:27:51.652337 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f38d3-d6c2-412a-bbd1-785dd95aa952 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:51.653167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f38d3-d6c2-412a-bbd1-785dd95aa952 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:51.653167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f38d3-d6c2-412a-bbd1-785dd95aa952 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:51.654593 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 746/1485] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:27:51.660558 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407600c-6ee3-4b5d-a7fb-9361de47ab1f None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:51.660967 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407600c-6ee3-4b5d-a7fb-9361de47ab1f None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:51.661114 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2407600c-6ee3-4b5d-a7fb-9361de47ab1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:51.679789 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-2407600c-6ee3-4b5d-a7fb-9361de47ab1f None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:27:51.680995 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-2407600c-6ee3-4b5d-a7fb-9361de47ab1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:27:51.688726 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 740/1486] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:27:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:27:51.726873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f38d3-d6c2-412a-bbd1-785dd95aa952 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:27:51.738369 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f38d3-d6c2-412a-bbd1-785dd95aa952 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:27:51.742079 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8124c6-4fa9-4499-98ef-6718d7eb5b62 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:27:51.742729 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8124c6-4fa9-4499-98ef-6718d7eb5b62 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:51.743018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8124c6-4fa9-4499-98ef-6718d7eb5b62 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:51.743319 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8124c6-4fa9-4499-98ef-6718d7eb5b62 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:51.752399 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8124c6-4fa9-4499-98ef-6718d7eb5b62 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:27:51.753869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8124c6-4fa9-4499-98ef-6718d7eb5b62 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:27:51.763406 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 747/1487] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:27:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:27:53.992191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f10265-5349-4576-9c28-c3a2a3f65610 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:53.992734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f10265-5349-4576-9c28-c3a2a3f65610 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:53.992734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f10265-5349-4576-9c28-c3a2a3f65610 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:53.994122 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 741/1488] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:27:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:27:53.999312 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80692b8e-9f26-428a-8185-ee0d0df4b018 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:53.999716 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80692b8e-9f26-428a-8185-ee0d0df4b018 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:53.999880 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80692b8e-9f26-428a-8185-ee0d0df4b018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:54.018095 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-80692b8e-9f26-428a-8185-ee0d0df4b018 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:27:54.019463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-80692b8e-9f26-428a-8185-ee0d0df4b018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:27:54.027161 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 748/1489] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:27:54.067835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f10265-5349-4576-9c28-c3a2a3f65610 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:27:54.078539 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f10265-5349-4576-9c28-c3a2a3f65610 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:27:54.082963 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cf2aa3-8dae-4697-b097-295ef5c84208 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:27:54.083628 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf2aa3-8dae-4697-b097-295ef5c84208 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:54.083916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf2aa3-8dae-4697-b097-295ef5c84208 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:54.084190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cf2aa3-8dae-4697-b097-295ef5c84208 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:54.092864 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf2aa3-8dae-4697-b097-295ef5c84208 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:27:54.094281 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cf2aa3-8dae-4697-b097-295ef5c84208 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:27:54.104597 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 742/1490] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:27:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:27:56.535725 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00221e-6d3f-4eef-8e34-8d67e7d59d89 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:56.536181 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00221e-6d3f-4eef-8e34-8d67e7d59d89 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:56.536306 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00221e-6d3f-4eef-8e34-8d67e7d59d89 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:56.537704 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 749/1491] 10.222.0.47 () {54 vars in 926 bytes} [Thu Jun 30 10:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:27:56.542787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46fbfa-ce8c-4b0f-b339-a2df6d05fe58 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:56.543207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46fbfa-ce8c-4b0f-b339-a2df6d05fe58 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:56.543382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea46fbfa-ce8c-4b0f-b339-a2df6d05fe58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:56.560721 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-ea46fbfa-ce8c-4b0f-b339-a2df6d05fe58 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:27:56.561915 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-ea46fbfa-ce8c-4b0f-b339-a2df6d05fe58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:27:56.570819 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 743/1492] 10.222.0.47 () {58 vars in 1023 bytes} [Thu Jun 30 10:27:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:27:56.610306 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00221e-6d3f-4eef-8e34-8d67e7d59d89 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:27:56.621212 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00221e-6d3f-4eef-8e34-8d67e7d59d89 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:27:56.624681 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0452815f-4ccf-41a4-9a46-cc9e9f65dd6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:27:56.625341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452815f-4ccf-41a4-9a46-cc9e9f65dd6f service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:27:56.625622 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452815f-4ccf-41a4-9a46-cc9e9f65dd6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:27:56.625897 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0452815f-4ccf-41a4-9a46-cc9e9f65dd6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:27:56.636368 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0452815f-4ccf-41a4-9a46-cc9e9f65dd6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:27:56.637796 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0452815f-4ccf-41a4-9a46-cc9e9f65dd6f service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:27:56.648103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 750/1493] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:27:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:28:06.555714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=33 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:28:06.644038 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=33 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:29:38.965193 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14787922-3d00-49fb-9bb6-aa81f6ac858d None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:29:38.965787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14787922-3d00-49fb-9bb6-aa81f6ac858d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:29:38.965787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14787922-3d00-49fb-9bb6-aa81f6ac858d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:29:38.983196 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-14787922-3d00-49fb-9bb6-aa81f6ac858d None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:29:38.984409 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-14787922-3d00-49fb-9bb6-aa81f6ac858d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:29:38.992679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 744/1494] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:29:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:29:39.000405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62076e75-910e-4910-8fbb-2938baef70f4 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:29:39.001540 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62076e75-910e-4910-8fbb-2938baef70f4 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:29:39.001640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62076e75-910e-4910-8fbb-2938baef70f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:29:39.019018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-62076e75-910e-4910-8fbb-2938baef70f4 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:29:39.020190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-62076e75-910e-4910-8fbb-2938baef70f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:29:39.027960 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 751/1495] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:29:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:29:39.032689 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14787922-3d00-49fb-9bb6-aa81f6ac858d None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:29:39.042385 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14787922-3d00-49fb-9bb6-aa81f6ac858d None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:29:39.046758 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4235e74-d04a-4ac9-ae0f-02e6a997082d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:29:39.048047 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4235e74-d04a-4ac9-ae0f-02e6a997082d admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:29:39.048290 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4235e74-d04a-4ac9-ae0f-02e6a997082d admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:29:39.048589 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4235e74-d04a-4ac9-ae0f-02e6a997082d admin admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:29:39.049296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4235e74-d04a-4ac9-ae0f-02e6a997082d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:29:39.049636 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4235e74-d04a-4ac9-ae0f-02e6a997082d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:29:39.050652 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: WARNING py.warnings [None req-e4235e74-d04a-4ac9-ae0f-02e6a997082d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 10:29:39.050652 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: warnings.warn(msg) Jun 30 10:29:39.050652 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: Jun 30 10:29:39.051559 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4235e74-d04a-4ac9-ae0f-02e6a997082d admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:29:39.062866 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 745/1496] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:29:39.071353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62076e75-910e-4910-8fbb-2938baef70f4 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:29:39.080668 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62076e75-910e-4910-8fbb-2938baef70f4 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:29:39.085188 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53400675-9643-4d12-b1f9-d09964e29417 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:29:39.085836 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53400675-9643-4d12-b1f9-d09964e29417 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:29:39.086122 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53400675-9643-4d12-b1f9-d09964e29417 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:29:39.086400 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53400675-9643-4d12-b1f9-d09964e29417 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:29:39.087101 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53400675-9643-4d12-b1f9-d09964e29417 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:29:39.088922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: WARNING py.warnings [None req-53400675-9643-4d12-b1f9-d09964e29417 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 10:29:39.088922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: warnings.warn(msg) Jun 30 10:29:39.088922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: Jun 30 10:29:39.090135 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53400675-9643-4d12-b1f9-d09964e29417 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:29:39.193893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 752/1497] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:29:39 2022] POST /identity/v3/projects => generated 377 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:29:39.198092 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4235e74-d04a-4ac9-ae0f-02e6a997082d admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:29:39.208197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4235e74-d04a-4ac9-ae0f-02e6a997082d admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:29:39.264991 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370fd475-c069-495b-91e1-df5219dbadf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:29:39.265698 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370fd475-c069-495b-91e1-df5219dbadf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:29:39.266004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370fd475-c069-495b-91e1-df5219dbadf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:29:39.266366 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370fd475-c069-495b-91e1-df5219dbadf2 admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:29:39.267068 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370fd475-c069-495b-91e1-df5219dbadf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:29:39.268112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: WARNING py.warnings [None req-370fd475-c069-495b-91e1-df5219dbadf2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 10:29:39.268112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: warnings.warn(msg) Jun 30 10:29:39.268112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: Jun 30 10:29:39.269375 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370fd475-c069-495b-91e1-df5219dbadf2 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:29:39.368543 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 746/1498] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:29:39 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:29:39.372660 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53400675-9643-4d12-b1f9-d09964e29417 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:29:39.382892 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53400675-9643-4d12-b1f9-d09964e29417 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:29:39.387579 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000d781-f2c1-42fc-8982-dd09f91e79ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:29:39.388236 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000d781-f2c1-42fc-8982-dd09f91e79ba admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:29:39.388523 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000d781-f2c1-42fc-8982-dd09f91e79ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:29:39.388802 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000d781-f2c1-42fc-8982-dd09f91e79ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:29:39.389529 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000d781-f2c1-42fc-8982-dd09f91e79ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:29:39.390568 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: WARNING py.warnings [None req-4000d781-f2c1-42fc-8982-dd09f91e79ba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 10:29:39.390568 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: warnings.warn(msg) Jun 30 10:29:39.390568 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: Jun 30 10:29:39.391616 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000d781-f2c1-42fc-8982-dd09f91e79ba admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:29:39.404889 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 753/1499] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:29:39.410014 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370fd475-c069-495b-91e1-df5219dbadf2 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:29:39.419755 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370fd475-c069-495b-91e1-df5219dbadf2 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:29:39.425311 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f1b33e-fe94-424d-aa21-72b7ee9a1d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:29:39.425964 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1b33e-fe94-424d-aa21-72b7ee9a1d52 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:29:39.426248 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1b33e-fe94-424d-aa21-72b7ee9a1d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:29:39.426523 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f1b33e-fe94-424d-aa21-72b7ee9a1d52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:29:39.427246 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f1b33e-fe94-424d-aa21-72b7ee9a1d52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:29:39.428280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: WARNING py.warnings [None req-56f1b33e-fe94-424d-aa21-72b7ee9a1d52 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 10:29:39.428280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: warnings.warn(msg) Jun 30 10:29:39.428280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: Jun 30 10:29:39.429165 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f1b33e-fe94-424d-aa21-72b7ee9a1d52 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:29:39.442684 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 747/1500] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:29:39.446642 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000d781-f2c1-42fc-8982-dd09f91e79ba admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:29:39.456760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000d781-f2c1-42fc-8982-dd09f91e79ba admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:29:39.461226 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a30016d-63b1-4737-999a-3d1ee6863f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:29:39.462095 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30016d-63b1-4737-999a-3d1ee6863f75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:29:39.462380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30016d-63b1-4737-999a-3d1ee6863f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:29:39.462659 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a30016d-63b1-4737-999a-3d1ee6863f75 admin admin] PATH_INFO: `/v3/projects/304f236bcd2d441baefd1caa1dce71a8/users/bd051dc9cbaf438cabca87ceb0d30127/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:29:39.477670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30016d-63b1-4737-999a-3d1ee6863f75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=304f236bcd2d441baefd1caa1dce71a8, user_id=bd051dc9cbaf438cabca87ceb0d30127, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:29:39.478703 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: WARNING py.warnings [None req-8a30016d-63b1-4737-999a-3d1ee6863f75 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 10:29:39.478703 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: warnings.warn(msg) Jun 30 10:29:39.478703 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: Jun 30 10:29:39.480698 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a30016d-63b1-4737-999a-3d1ee6863f75 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:29:39.586365 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 754/1501] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:29:39 2022] PUT /identity/v3/projects/304f236bcd2d441baefd1caa1dce71a8/users/bd051dc9cbaf438cabca87ceb0d30127/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:29:39.598929 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e007b6c-4fef-497c-b503-8f593764e6dc None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:29:39.599217 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e007b6c-4fef-497c-b503-8f593764e6dc None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:29:39.599528 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e007b6c-4fef-497c-b503-8f593764e6dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:29:39.625856 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-3e007b6c-4fef-497c-b503-8f593764e6dc None None] MFA Rules not processed for user `bd051dc9cbaf438cabca87ceb0d30127`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:29:39.627083 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-3e007b6c-4fef-497c-b503-8f593764e6dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:29:39.689767 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 748/1502] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:29:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:29:39.702662 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cb754-b385-4014-b19f-2e284da8076c None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:29:39.702951 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cb754-b385-4014-b19f-2e284da8076c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:29:39.703230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1cb754-b385-4014-b19f-2e284da8076c None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:29:39.705407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 755/1503] 10.222.0.47 () {54 vars in 963 bytes} [Thu Jun 30 10:29:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:29:39.710409 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6d289-7ecb-4f48-bd69-fd6c240e007f None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:29:39.710695 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6d289-7ecb-4f48-bd69-fd6c240e007f None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:29:39.710969 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e6d289-7ecb-4f48-bd69-fd6c240e007f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:29:39.729181 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-65e6d289-7ecb-4f48-bd69-fd6c240e007f None None] MFA Rules not processed for user `07ab12e5d1d94a009c4d414e5c43ac92`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:29:39.730371 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-65e6d289-7ecb-4f48-bd69-fd6c240e007f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:29:39.773352 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:29:39.780708 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 749/1504] 10.222.0.47 () {58 vars in 1060 bytes} [Thu Jun 30 10:29:39 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:29:39.790730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1cb754-b385-4014-b19f-2e284da8076c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:29:39.799773 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1cb754-b385-4014-b19f-2e284da8076c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:29:39.803526 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9a5ce-e86b-455e-b458-1830e33e5b3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:29:39.804164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9a5ce-e86b-455e-b458-1830e33e5b3e service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:29:39.804448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9a5ce-e86b-455e-b458-1830e33e5b3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:29:39.804724 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9a5ce-e86b-455e-b458-1830e33e5b3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:29:39.813829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab9a5ce-e86b-455e-b458-1830e33e5b3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:29:39.817616 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab9a5ce-e86b-455e-b458-1830e33e5b3e service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:29:39.823928 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 756/1505] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:29:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:29:39.866221 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6ba06-1ce2-4248-a154-d84a71e93098 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:29:39.866532 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6ba06-1ce2-4248-a154-d84a71e93098 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:29:39.866808 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6ba06-1ce2-4248-a154-d84a71e93098 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:29:39.868237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 750/1506] 10.222.0.47 () {52 vars in 829 bytes} [Thu Jun 30 10:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:29:39.871710 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9a5ce-e86b-455e-b458-1830e33e5b3e service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:29:39.933923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9a5ce-e86b-455e-b458-1830e33e5b3e service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:29:39.938457 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d2c82-b222-40de-9e23-59169f1455e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:29:39.939340 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d2c82-b222-40de-9e23-59169f1455e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:29:39.939628 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d2c82-b222-40de-9e23-59169f1455e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:29:39.939907 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34d2c82-b222-40de-9e23-59169f1455e9 admin admin] PATH_INFO: `/v3/users/bd051dc9cbaf438cabca87ceb0d30127` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:29:39.941110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34d2c82-b222-40de-9e23-59169f1455e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd051dc9cbaf438cabca87ceb0d30127)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:29:39.942125 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: WARNING py.warnings [None req-d34d2c82-b222-40de-9e23-59169f1455e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 10:29:39.942125 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: warnings.warn(msg) Jun 30 10:29:39.942125 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: Jun 30 10:29:39.943379 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34d2c82-b222-40de-9e23-59169f1455e9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:29:40.135337 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d34d2c82-b222-40de-9e23-59169f1455e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd051dc9cbaf438cabca87ceb0d30127'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:29:40.135962 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d34d2c82-b222-40de-9e23-59169f1455e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd051dc9cbaf438cabca87ceb0d30127'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:29:40.149546 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d34d2c82-b222-40de-9e23-59169f1455e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd051dc9cbaf438cabca87ceb0d30127'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:29:40.150363 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d34d2c82-b222-40de-9e23-59169f1455e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd051dc9cbaf438cabca87ceb0d30127'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:29:40.231381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d34d2c82-b222-40de-9e23-59169f1455e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd051dc9cbaf438cabca87ceb0d30127'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:29:40.249192 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:29:40.257881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 757/1507] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:29:39 2022] DELETE /identity/v3/users/bd051dc9cbaf438cabca87ceb0d30127 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:29:40.262324 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8c395-131f-44f4-a4c6-447b815d081a None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:29:40.262800 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8c395-131f-44f4-a4c6-447b815d081a None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:29:40.262908 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8c395-131f-44f4-a4c6-447b815d081a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:29:40.281951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-f5b8c395-131f-44f4-a4c6-447b815d081a None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:29:40.283349 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-f5b8c395-131f-44f4-a4c6-447b815d081a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:29:40.342004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 751/1508] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:29:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:29:40.351940 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d2c82-b222-40de-9e23-59169f1455e9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:29:40.353574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d34d2c82-b222-40de-9e23-59169f1455e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:29:40.410322 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34d2c82-b222-40de-9e23-59169f1455e9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:29:40.413849 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9bb50d-4397-4e90-94a1-39fc8b62313e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:29:40.414496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bb50d-4397-4e90-94a1-39fc8b62313e service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:29:40.414782 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bb50d-4397-4e90-94a1-39fc8b62313e service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:29:40.415062 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9bb50d-4397-4e90-94a1-39fc8b62313e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:29:40.424779 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9bb50d-4397-4e90-94a1-39fc8b62313e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:29:40.426226 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9bb50d-4397-4e90-94a1-39fc8b62313e service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:29:40.435901 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 758/1509] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:29:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:29:40.941752 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8c395-131f-44f4-a4c6-447b815d081a None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:29:40.943620 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-f5b8c395-131f-44f4-a4c6-447b815d081a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:29:40.957184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8c395-131f-44f4-a4c6-447b815d081a None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:29:40.962014 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde3d59d-0ea6-44e4-bae1-03400425473e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:29:40.962974 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3d59d-0ea6-44e4-bae1-03400425473e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:29:40.963258 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3d59d-0ea6-44e4-bae1-03400425473e admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:29:40.963637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3d59d-0ea6-44e4-bae1-03400425473e admin admin] PATH_INFO: `/v3/projects/304f236bcd2d441baefd1caa1dce71a8` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:29:40.964825 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde3d59d-0ea6-44e4-bae1-03400425473e admin admin] RBAC: Authorizing `identity:delete_project(project_id=304f236bcd2d441baefd1caa1dce71a8)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:29:40.965967 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: WARNING py.warnings [None req-bde3d59d-0ea6-44e4-bae1-03400425473e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 10:29:40.965967 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: warnings.warn(msg) Jun 30 10:29:40.965967 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: Jun 30 10:29:40.967217 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde3d59d-0ea6-44e4-bae1-03400425473e admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:29:41.081450 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bde3d59d-0ea6-44e4-bae1-03400425473e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '304f236bcd2d441baefd1caa1dce71a8', 'request_id': 'req-bde3d59d-0ea6-44e4-bae1-03400425473e'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:29:41.095337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bde3d59d-0ea6-44e4-bae1-03400425473e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '304f236bcd2d441baefd1caa1dce71a8', 'request_id': 'req-bde3d59d-0ea6-44e4-bae1-03400425473e'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:29:41.101405 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bde3d59d-0ea6-44e4-bae1-03400425473e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '304f236bcd2d441baefd1caa1dce71a8', 'request_id': 'req-bde3d59d-0ea6-44e4-bae1-03400425473e'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:29:41.174205 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bde3d59d-0ea6-44e4-bae1-03400425473e admin admin] The token cache is being invalidate because project 304f236bcd2d441baefd1caa1dce71a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:29:41.174495 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bde3d59d-0ea6-44e4-bae1-03400425473e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:29:41.175220 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bde3d59d-0ea6-44e4-bae1-03400425473e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:29:41.176858 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 752/1510] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:29:40 2022] DELETE /identity/v3/projects/304f236bcd2d441baefd1caa1dce71a8 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:29:50.431622 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:29:51.170650 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:31:22.623702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd3f5c-6ea5-4dcf-b765-9f006e44f97b None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:22.624621 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd3f5c-6ea5-4dcf-b765-9f006e44f97b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:22.624621 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd3f5c-6ea5-4dcf-b765-9f006e44f97b None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:22.626297 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 10:31:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:31:22.632807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f1023-140d-4f70-8269-6ab4d0eecedd None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:22.633258 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f1023-140d-4f70-8269-6ab4d0eecedd None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:22.633365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f1023-140d-4f70-8269-6ab4d0eecedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:22.651987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-839f1023-140d-4f70-8269-6ab4d0eecedd None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:31:22.653208 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-839f1023-140d-4f70-8269-6ab4d0eecedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:31:22.715736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 10:31:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:31:22.872082 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd3f5c-6ea5-4dcf-b765-9f006e44f97b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:31:22.874067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-26cd3f5c-6ea5-4dcf-b765-9f006e44f97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:31:22.956133 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd3f5c-6ea5-4dcf-b765-9f006e44f97b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:31:22.965963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad8e86-ac0f-4c0c-b9b2-6a44d72ed93d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:31:22.966625 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad8e86-ac0f-4c0c-b9b2-6a44d72ed93d service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:22.966911 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad8e86-ac0f-4c0c-b9b2-6a44d72ed93d service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:22.967190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad8e86-ac0f-4c0c-b9b2-6a44d72ed93d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:22.976858 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad8e86-ac0f-4c0c-b9b2-6a44d72ed93d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:31:22.978488 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dad8e86-ac0f-4c0c-b9b2-6a44d72ed93d service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:31:22.988625 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 760/1513] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:31:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:31:23.701469 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a237f-605f-4740-993a-2e169960b109 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:23.702052 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a237f-605f-4740-993a-2e169960b109 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:23.702295 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a237f-605f-4740-993a-2e169960b109 None None] PATH_INFO: `/v3` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:23.704403 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 754/1514] 10.222.0.15 () {54 vars in 925 bytes} [Thu Jun 30 10:31:23 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:31:23.713587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae662560-c699-4a8c-8a8f-8cb0b5c68c35 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:23.714085 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae662560-c699-4a8c-8a8f-8cb0b5c68c35 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:23.714231 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae662560-c699-4a8c-8a8f-8cb0b5c68c35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:23.733053 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ae662560-c699-4a8c-8a8f-8cb0b5c68c35 None None] MFA Rules not processed for user `46c99f43a9344d7d92632990e45bb9cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:31:23.734270 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ae662560-c699-4a8c-8a8f-8cb0b5c68c35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:31:23.799912 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 761/1515] 10.222.0.15 () {58 vars in 1015 bytes} [Thu Jun 30 10:31:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:31:23.852546 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c60963-0f0b-4ef6-8c25-3d46a284d32e None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:23.852836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c60963-0f0b-4ef6-8c25-3d46a284d32e None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:23.853117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c60963-0f0b-4ef6-8c25-3d46a284d32e None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:23.854489 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 755/1516] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 10:31:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:31:23.859434 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d4583-7033-4ee1-9ffa-79f563b516aa None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:23.859731 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d4583-7033-4ee1-9ffa-79f563b516aa None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:23.860014 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d4583-7033-4ee1-9ffa-79f563b516aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:23.882004 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-794d4583-7033-4ee1-9ffa-79f563b516aa None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:31:23.883217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-794d4583-7033-4ee1-9ffa-79f563b516aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:31:23.891175 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 762/1517] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 10:31:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:31:24.046883 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c60963-0f0b-4ef6-8c25-3d46a284d32e None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:31:24.061888 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c60963-0f0b-4ef6-8c25-3d46a284d32e None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:31:24.067763 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920578e-9a7c-4ba9-906c-058e51c6f470 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:31:24.068568 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920578e-9a7c-4ba9-906c-058e51c6f470 service nova] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:24.068858 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920578e-9a7c-4ba9-906c-058e51c6f470 service nova] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:24.069140 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920578e-9a7c-4ba9-906c-058e51c6f470 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:24.077481 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920578e-9a7c-4ba9-906c-058e51c6f470 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:31:24.079119 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920578e-9a7c-4ba9-906c-058e51c6f470 service nova] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:31:24.089481 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 756/1518] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:31:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:31:32.764425 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81da6b-a36d-4065-9dad-717b8eb2db8d None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:32.764969 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81da6b-a36d-4065-9dad-717b8eb2db8d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:32.764969 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d81da6b-a36d-4065-9dad-717b8eb2db8d None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:32.766263 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 763/1519] 10.222.0.47 () {54 vars in 960 bytes} [Thu Jun 30 10:31:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:31:32.771427 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cfb2f-399d-4f5a-83f0-f94517cde223 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:32.771831 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cfb2f-399d-4f5a-83f0-f94517cde223 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:32.771991 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cfb2f-399d-4f5a-83f0-f94517cde223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:32.790669 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-919cfb2f-399d-4f5a-83f0-f94517cde223 None None] MFA Rules not processed for user `46c99f43a9344d7d92632990e45bb9cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:31:32.792584 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-919cfb2f-399d-4f5a-83f0-f94517cde223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:31:32.800662 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 757/1520] 10.222.0.47 () {58 vars in 1057 bytes} [Thu Jun 30 10:31:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:31:32.808593 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81da6b-a36d-4065-9dad-717b8eb2db8d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:31:32.822380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d81da6b-a36d-4065-9dad-717b8eb2db8d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:31:32.827436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a39f67-2d6d-4a8b-bf56-8f490fb3abfc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46c99f43a9344d7d92632990e45bb9cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:31:32.828076 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a39f67-2d6d-4a8b-bf56-8f490fb3abfc service placement] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:32.828361 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a39f67-2d6d-4a8b-bf56-8f490fb3abfc service placement] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:32.828639 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a39f67-2d6d-4a8b-bf56-8f490fb3abfc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:32.837199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a39f67-2d6d-4a8b-bf56-8f490fb3abfc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:31:32.838844 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a39f67-2d6d-4a8b-bf56-8f490fb3abfc service placement] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:31:32.849449 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 764/1521] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:31:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:31:35.365925 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21572c1-834b-4e02-a364-532f8dc605bb None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:35.366531 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21572c1-834b-4e02-a364-532f8dc605bb None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:35.366531 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21572c1-834b-4e02-a364-532f8dc605bb None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:35.367878 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 10:31:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:31:35.372599 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fc512-41fa-478e-ba1f-d05755b95628 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:35.373026 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fc512-41fa-478e-ba1f-d05755b95628 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:35.373160 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fc512-41fa-478e-ba1f-d05755b95628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:35.391581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f44fc512-41fa-478e-ba1f-d05755b95628 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:31:35.392785 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f44fc512-41fa-478e-ba1f-d05755b95628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:31:35.401371 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 10:31:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:31:35.555334 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21572c1-834b-4e02-a364-532f8dc605bb None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:31:35.570227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21572c1-834b-4e02-a364-532f8dc605bb None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:31:35.575560 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03871f28-bcb3-4b5f-9f6f-9726d94dfa50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:31:35.576200 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03871f28-bcb3-4b5f-9f6f-9726d94dfa50 service nova] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:35.576497 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03871f28-bcb3-4b5f-9f6f-9726d94dfa50 service nova] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:35.576770 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03871f28-bcb3-4b5f-9f6f-9726d94dfa50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:35.584964 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03871f28-bcb3-4b5f-9f6f-9726d94dfa50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:31:35.586564 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03871f28-bcb3-4b5f-9f6f-9726d94dfa50 service nova] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:31:35.595864 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 759/1524] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:31:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:31:36.421299 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73801ba6-985a-4ba0-9246-33a116612f2f None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:36.421878 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73801ba6-985a-4ba0-9246-33a116612f2f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:36.421878 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73801ba6-985a-4ba0-9246-33a116612f2f None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:36.423313 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 30 10:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:31:36.428627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39e2e6-3ed7-4a29-b364-a45bbd60077a None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:36.429071 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39e2e6-3ed7-4a29-b364-a45bbd60077a None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:36.429180 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d39e2e6-3ed7-4a29-b364-a45bbd60077a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:36.447487 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-0d39e2e6-3ed7-4a29-b364-a45bbd60077a None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:31:36.448695 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-0d39e2e6-3ed7-4a29-b364-a45bbd60077a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:31:36.456347 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 30 10:31:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:31:36.609490 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73801ba6-985a-4ba0-9246-33a116612f2f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:31:36.623775 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73801ba6-985a-4ba0-9246-33a116612f2f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:31:36.629251 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fb8bda-02ee-4de2-9d53-8e0566cd7ba1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:31:36.629896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb8bda-02ee-4de2-9d53-8e0566cd7ba1 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:36.630774 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb8bda-02ee-4de2-9d53-8e0566cd7ba1 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:36.631224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fb8bda-02ee-4de2-9d53-8e0566cd7ba1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:36.639179 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb8bda-02ee-4de2-9d53-8e0566cd7ba1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:31:36.640789 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fb8bda-02ee-4de2-9d53-8e0566cd7ba1 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:31:36.651017 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 767/1527] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:31:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:31:40.586845 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d1868-0d33-4bbe-aeaf-a9d08ed2391a None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:40.587447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d1868-0d33-4bbe-aeaf-a9d08ed2391a None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:40.587447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d1868-0d33-4bbe-aeaf-a9d08ed2391a None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:40.588844 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 10:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:31:40.594255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16edd8-5eef-450d-a4c4-e83f3021339f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:40.594696 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16edd8-5eef-450d-a4c4-e83f3021339f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:40.594817 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16edd8-5eef-450d-a4c4-e83f3021339f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:40.612406 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3f16edd8-5eef-450d-a4c4-e83f3021339f None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:31:40.613615 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3f16edd8-5eef-450d-a4c4-e83f3021339f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:31:40.621905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 10:31:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:31:40.658636 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d1868-0d33-4bbe-aeaf-a9d08ed2391a None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:31:40.660427 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-3f9d1868-0d33-4bbe-aeaf-a9d08ed2391a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:31:40.717635 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d1868-0d33-4bbe-aeaf-a9d08ed2391a None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:31:40.721249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee386b2-8167-4fca-9789-178f058b4fd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:31:40.721894 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee386b2-8167-4fca-9789-178f058b4fd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:40.722183 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee386b2-8167-4fca-9789-178f058b4fd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:40.722465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee386b2-8167-4fca-9789-178f058b4fd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:40.731577 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee386b2-8167-4fca-9789-178f058b4fd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:31:40.733013 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee386b2-8167-4fca-9789-178f058b4fd0 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:31:40.743223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 762/1530] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:31:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:31:42.762919 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a2450-d536-43de-b9d9-65f26a85b400 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:42.763492 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a2450-d536-43de-b9d9-65f26a85b400 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:42.763492 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a2450-d536-43de-b9d9-65f26a85b400 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:42.764816 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 10:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:31:42.770000 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc04194-552c-4527-b522-c9cc0770720d None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:42.770443 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc04194-552c-4527-b522-c9cc0770720d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:42.770872 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc04194-552c-4527-b522-c9cc0770720d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:42.791007 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-fcc04194-552c-4527-b522-c9cc0770720d None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:31:42.792206 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-fcc04194-552c-4527-b522-c9cc0770720d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:31:42.800091 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 10:31:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:31:42.837669 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a2450-d536-43de-b9d9-65f26a85b400 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:31:42.849246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a2450-d536-43de-b9d9-65f26a85b400 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:31:42.852835 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c83b54d-8aeb-4bfa-9f9c-28c4a9c641ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:31:42.853473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83b54d-8aeb-4bfa-9f9c-28c4a9c641ef service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:42.853762 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83b54d-8aeb-4bfa-9f9c-28c4a9c641ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:42.854039 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c83b54d-8aeb-4bfa-9f9c-28c4a9c641ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:42.863349 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c83b54d-8aeb-4bfa-9f9c-28c4a9c641ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:31:42.864869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c83b54d-8aeb-4bfa-9f9c-28c4a9c641ef service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:31:42.874768 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 770/1533] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:31:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:31:46.497532 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c4925-6b58-45fb-a53a-41b88b5e076e None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:46.498077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c4925-6b58-45fb-a53a-41b88b5e076e None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:46.498077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16c4925-6b58-45fb-a53a-41b88b5e076e None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:46.499496 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 10:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:31:46.504959 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b960450-84a8-4ca2-b8e5-bb1a9934f03d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:46.505350 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b960450-84a8-4ca2-b8e5-bb1a9934f03d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:46.505525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b960450-84a8-4ca2-b8e5-bb1a9934f03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:46.523410 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8b960450-84a8-4ca2-b8e5-bb1a9934f03d None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:31:46.524602 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8b960450-84a8-4ca2-b8e5-bb1a9934f03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:31:46.532683 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 10:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:31:46.833920 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c4925-6b58-45fb-a53a-41b88b5e076e None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:31:46.836577 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-b16c4925-6b58-45fb-a53a-41b88b5e076e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:31:46.897368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16c4925-6b58-45fb-a53a-41b88b5e076e None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:31:46.901003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ace55-6bf4-444f-aafc-23ba89a66e6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:31:46.901682 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ace55-6bf4-444f-aafc-23ba89a66e6c service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:46.901946 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ace55-6bf4-444f-aafc-23ba89a66e6c service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:46.902221 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ace55-6bf4-444f-aafc-23ba89a66e6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:46.912572 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ace55-6bf4-444f-aafc-23ba89a66e6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:31:46.913993 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ace55-6bf4-444f-aafc-23ba89a66e6c service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:31:46.918904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:31:46.923584 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 765/1536] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:31:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:31:48.996922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78f967-56a2-45ab-b8db-4551c361abfb None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:48.997485 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78f967-56a2-45ab-b8db-4551c361abfb None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:48.997485 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd78f967-56a2-45ab-b8db-4551c361abfb None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:48.998858 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 30 10:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:31:49.003967 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3717a0c-61f5-4d95-adf7-ee884486de3a None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:49.004450 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3717a0c-61f5-4d95-adf7-ee884486de3a None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:49.004769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3717a0c-61f5-4d95-adf7-ee884486de3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:49.024691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-a3717a0c-61f5-4d95-adf7-ee884486de3a None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:31:49.025881 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-a3717a0c-61f5-4d95-adf7-ee884486de3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:31:49.033837 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 30 10:31:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:31:49.341550 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78f967-56a2-45ab-b8db-4551c361abfb None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:31:49.343340 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bd78f967-56a2-45ab-b8db-4551c361abfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:31:49.356329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd78f967-56a2-45ab-b8db-4551c361abfb None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:31:49.359850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc060f6d-4132-4ba4-bdc4-1d6067adf2df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:31:49.360486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc060f6d-4132-4ba4-bdc4-1d6067adf2df service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:31:49.360774 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc060f6d-4132-4ba4-bdc4-1d6067adf2df service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:31:49.361055 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc060f6d-4132-4ba4-bdc4-1d6067adf2df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:31:49.369974 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc060f6d-4132-4ba4-bdc4-1d6067adf2df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:31:49.371382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc060f6d-4132-4ba4-bdc4-1d6067adf2df service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:31:49.381445 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 773/1539] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:31:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:31:59.017247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:31:59.376616 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:18.834895 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17d6dc-ca5d-41a3-bf44-8e2d4ee7edec None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:18.836880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17d6dc-ca5d-41a3-bf44-8e2d4ee7edec None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:18.836880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17d6dc-ca5d-41a3-bf44-8e2d4ee7edec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:18.863306 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-7d17d6dc-ca5d-41a3-bf44-8e2d4ee7edec None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:18.864926 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-7d17d6dc-ca5d-41a3-bf44-8e2d4ee7edec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:18.881084 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 767/1540] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:18.898044 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7989e4-35d3-4358-996a-16b45c802888 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:18.898044 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7989e4-35d3-4358-996a-16b45c802888 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:18.898044 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7989e4-35d3-4358-996a-16b45c802888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:18.917559 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69395872-1bda-4e1e-b878-0a0b6bc4f894 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:18.922218 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69395872-1bda-4e1e-b878-0a0b6bc4f894 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:18.922218 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69395872-1bda-4e1e-b878-0a0b6bc4f894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:18.922446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6a7989e4-35d3-4358-996a-16b45c802888 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:18.923084 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6a7989e4-35d3-4358-996a-16b45c802888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:18.938843 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 774/1541] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:18.952713 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7989e4-35d3-4358-996a-16b45c802888 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:18.952981 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-69395872-1bda-4e1e-b878-0a0b6bc4f894 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:18.954754 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-69395872-1bda-4e1e-b878-0a0b6bc4f894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:18.969012 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7989e4-35d3-4358-996a-16b45c802888 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:18.971104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 768/1542] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:33:18.982732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98361415-691e-457d-8854-8971fcda2c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:18.983302 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8036b5-96d7-40cf-859e-1d19316aefc1 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:18.983658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8036b5-96d7-40cf-859e-1d19316aefc1 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:18.984158 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98361415-691e-457d-8854-8971fcda2c57 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:18.984451 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8036b5-96d7-40cf-859e-1d19316aefc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:18.985138 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98361415-691e-457d-8854-8971fcda2c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:18.985619 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98361415-691e-457d-8854-8971fcda2c57 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:18.986565 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98361415-691e-457d-8854-8971fcda2c57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:33:18.987427 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98361415-691e-457d-8854-8971fcda2c57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:18.988685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: WARNING py.warnings [None req-98361415-691e-457d-8854-8971fcda2c57 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 30 10:33:18.988685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: warnings.warn(msg) Jun 30 10:33:18.988685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: Jun 30 10:33:18.989957 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98361415-691e-457d-8854-8971fcda2c57 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:19.004518 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 775/1543] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:33:19.006152 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa255b-2f97-4d18-8449-80e91008beff None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.006683 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa255b-2f97-4d18-8449-80e91008beff None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.007184 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64aa255b-2f97-4d18-8449-80e91008beff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.020267 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-1f8036b5-96d7-40cf-859e-1d19316aefc1 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:19.021832 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-1f8036b5-96d7-40cf-859e-1d19316aefc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:19.037032 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 769/1544] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:19.038236 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8036b5-96d7-40cf-859e-1d19316aefc1 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:19.043661 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-64aa255b-2f97-4d18-8449-80e91008beff None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:19.044885 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-64aa255b-2f97-4d18-8449-80e91008beff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:19.062852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8036b5-96d7-40cf-859e-1d19316aefc1 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:19.073392 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 776/1545] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:19.074348 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa255b-2f97-4d18-8449-80e91008beff None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:19.074625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d360f51-8059-4819-84fa-2316d2ccbbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:19.075582 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d360f51-8059-4819-84fa-2316d2ccbbe4 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.076100 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d360f51-8059-4819-84fa-2316d2ccbbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.076613 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d360f51-8059-4819-84fa-2316d2ccbbe4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.077535 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d360f51-8059-4819-84fa-2316d2ccbbe4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:19.078782 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: WARNING py.warnings [None req-6d360f51-8059-4819-84fa-2316d2ccbbe4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 10:33:19.078782 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: warnings.warn(msg) Jun 30 10:33:19.078782 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: Jun 30 10:33:19.085518 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d360f51-8059-4819-84fa-2316d2ccbbe4 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:19.088982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64aa255b-2f97-4d18-8449-80e91008beff None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:19.106054 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e15e9-6caf-427e-ba9b-f23960733709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:19.106744 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e15e9-6caf-427e-ba9b-f23960733709 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.107037 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e15e9-6caf-427e-ba9b-f23960733709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.107336 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e15e9-6caf-427e-ba9b-f23960733709 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.108088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e15e9-6caf-427e-ba9b-f23960733709 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:33:19.108400 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e15e9-6caf-427e-ba9b-f23960733709 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:19.110031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e15e9-6caf-427e-ba9b-f23960733709 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:19.133206 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 777/1546] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:33:19.134456 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b55bec-9b9d-4b46-bdb0-ba1f4226d787 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.134786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b55bec-9b9d-4b46-bdb0-ba1f4226d787 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.135073 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b55bec-9b9d-4b46-bdb0-ba1f4226d787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.162494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-97b55bec-9b9d-4b46-bdb0-ba1f4226d787 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:19.163750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-97b55bec-9b9d-4b46-bdb0-ba1f4226d787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:19.183119 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 778/1547] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:19.185841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544ae25-6121-49f2-aa10-d0a40b621362 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.186346 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544ae25-6121-49f2-aa10-d0a40b621362 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.186822 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f544ae25-6121-49f2-aa10-d0a40b621362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.208492 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 770/1548] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:33:19 2022] POST /identity/v3/projects => generated 375 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:19.210863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2d4ac-ac52-4975-9b02-d823b8ffe8e9 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.210863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2d4ac-ac52-4975-9b02-d823b8ffe8e9 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.211346 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa2d4ac-ac52-4975-9b02-d823b8ffe8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.214023 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f544ae25-6121-49f2-aa10-d0a40b621362 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:19.215485 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f544ae25-6121-49f2-aa10-d0a40b621362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:19.231936 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-ffa2d4ac-ac52-4975-9b02-d823b8ffe8e9 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:19.233140 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-ffa2d4ac-ac52-4975-9b02-d823b8ffe8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:19.281743 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 779/1549] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:19.282881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544ae25-6121-49f2-aa10-d0a40b621362 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:19.293367 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f544ae25-6121-49f2-aa10-d0a40b621362 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:19.298440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd143a05-ccd2-47a5-a1fd-d8253c020115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:19.301972 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd143a05-ccd2-47a5-a1fd-d8253c020115 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.301972 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd143a05-ccd2-47a5-a1fd-d8253c020115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.301972 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd143a05-ccd2-47a5-a1fd-d8253c020115 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.301972 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd143a05-ccd2-47a5-a1fd-d8253c020115 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:19.302341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd143a05-ccd2-47a5-a1fd-d8253c020115 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:19.312914 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 771/1550] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:19.314855 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168385d8-24a8-48ff-b36d-b2ff655d7433 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.315323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168385d8-24a8-48ff-b36d-b2ff655d7433 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.315606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168385d8-24a8-48ff-b36d-b2ff655d7433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.334960 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-168385d8-24a8-48ff-b36d-b2ff655d7433 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:19.336273 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-168385d8-24a8-48ff-b36d-b2ff655d7433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:19.344547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 772/1551] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:19.345318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168385d8-24a8-48ff-b36d-b2ff655d7433 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:19.355876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168385d8-24a8-48ff-b36d-b2ff655d7433 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:19.360541 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974c614-fc18-4d43-b195-e4369662fa45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:19.361198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974c614-fc18-4d43-b195-e4369662fa45 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.361480 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974c614-fc18-4d43-b195-e4369662fa45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.361753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3974c614-fc18-4d43-b195-e4369662fa45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.362534 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974c614-fc18-4d43-b195-e4369662fa45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:33:19.362831 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974c614-fc18-4d43-b195-e4369662fa45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:19.364447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3974c614-fc18-4d43-b195-e4369662fa45 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:19.377317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 773/1552] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:33:19.378095 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974c614-fc18-4d43-b195-e4369662fa45 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:19.388667 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 780/1553] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:33:19 2022] POST /identity/v3/projects => generated 403 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:19.391859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3974c614-fc18-4d43-b195-e4369662fa45 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:19.392015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0afe50-1a8e-497d-ae15-7aaa47ea6acd None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.392015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0afe50-1a8e-497d-ae15-7aaa47ea6acd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.392015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0afe50-1a8e-497d-ae15-7aaa47ea6acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.409891 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-cc0afe50-1a8e-497d-ae15-7aaa47ea6acd None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:19.411137 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cc0afe50-1a8e-497d-ae15-7aaa47ea6acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:19.448238 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e609b0c-d4bf-4604-9fc1-5ab7beb75a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:19.448925 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e609b0c-d4bf-4604-9fc1-5ab7beb75a43 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.449212 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e609b0c-d4bf-4604-9fc1-5ab7beb75a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.449492 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e609b0c-d4bf-4604-9fc1-5ab7beb75a43 admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.450185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e609b0c-d4bf-4604-9fc1-5ab7beb75a43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:19.452617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e609b0c-d4bf-4604-9fc1-5ab7beb75a43 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:19.472928 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 781/1554] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:19.474300 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ae98e-491b-4f05-be50-e0bf5728517e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.474584 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ae98e-491b-4f05-be50-e0bf5728517e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.474856 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219ae98e-491b-4f05-be50-e0bf5728517e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.493240 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-219ae98e-491b-4f05-be50-e0bf5728517e None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:19.495136 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-219ae98e-491b-4f05-be50-e0bf5728517e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:19.503055 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 782/1555] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:19.504544 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6389e22-24fa-405e-97c1-e6f4718681d7 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.504669 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6389e22-24fa-405e-97c1-e6f4718681d7 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.504838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6389e22-24fa-405e-97c1-e6f4718681d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.523437 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a6389e22-24fa-405e-97c1-e6f4718681d7 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:19.524613 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a6389e22-24fa-405e-97c1-e6f4718681d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:19.530111 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 774/1556] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:33:19 2022] POST /identity/v3/users => generated 288 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:19.531502 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c7a51-0d93-48d8-8f0e-7f0060d65dc7 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.531932 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c7a51-0d93-48d8-8f0e-7f0060d65dc7 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.532219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c7a51-0d93-48d8-8f0e-7f0060d65dc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.533463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 783/1557] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:19.534670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adab6e8-41d9-474d-a329-ad4c4590e52c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.534955 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adab6e8-41d9-474d-a329-ad4c4590e52c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.535227 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adab6e8-41d9-474d-a329-ad4c4590e52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.552701 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8adab6e8-41d9-474d-a329-ad4c4590e52c None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:19.553885 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8adab6e8-41d9-474d-a329-ad4c4590e52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:19.555761 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-6b0c7a51-0d93-48d8-8f0e-7f0060d65dc7 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:19.557491 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-6b0c7a51-0d93-48d8-8f0e-7f0060d65dc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:19.562242 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 784/1558] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:19.562983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adab6e8-41d9-474d-a329-ad4c4590e52c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:19.569021 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 775/1559] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:19.569021 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c7a51-0d93-48d8-8f0e-7f0060d65dc7 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:19.575550 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adab6e8-41d9-474d-a329-ad4c4590e52c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:19.576214 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c7a51-0d93-48d8-8f0e-7f0060d65dc7 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:19.580128 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991cbbe1-5891-4cc1-aacc-6b6203e8c734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:19.580654 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe27b7e-217a-48a7-a086-ee9a30f442ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:19.580787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991cbbe1-5891-4cc1-aacc-6b6203e8c734 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.581165 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991cbbe1-5891-4cc1-aacc-6b6203e8c734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.581301 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe27b7e-217a-48a7-a086-ee9a30f442ef admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.581448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991cbbe1-5891-4cc1-aacc-6b6203e8c734 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.581591 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe27b7e-217a-48a7-a086-ee9a30f442ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.581871 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe27b7e-217a-48a7-a086-ee9a30f442ef admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.582131 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991cbbe1-5891-4cc1-aacc-6b6203e8c734 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:19.582552 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe27b7e-217a-48a7-a086-ee9a30f442ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:19.584083 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991cbbe1-5891-4cc1-aacc-6b6203e8c734 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:19.584496 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe27b7e-217a-48a7-a086-ee9a30f442ef admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:19.594280 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:19.672748 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 776/1560] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:33:19 2022] POST /identity/v3/users => generated 302 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:19.673624 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 785/1561] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:33:19 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:19.674459 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991cbbe1-5891-4cc1-aacc-6b6203e8c734 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:19.674644 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d4f67-f137-4309-ad11-af75c1c6a32c None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.675084 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d4f67-f137-4309-ad11-af75c1c6a32c None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.675411 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6d4f67-f137-4309-ad11-af75c1c6a32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.683938 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:19.694362 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-cc6d4f67-f137-4309-ad11-af75c1c6a32c None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:19.695601 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-cc6d4f67-f137-4309-ad11-af75c1c6a32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:19.739852 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991cbbe1-5891-4cc1-aacc-6b6203e8c734 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:19.744634 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf0796-3f77-413f-aae8-ed3843122a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:19.745879 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf0796-3f77-413f-aae8-ed3843122a68 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.746165 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf0796-3f77-413f-aae8-ed3843122a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.746451 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf0796-3f77-413f-aae8-ed3843122a68 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.747179 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf0796-3f77-413f-aae8-ed3843122a68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:33:19.747521 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf0796-3f77-413f-aae8-ed3843122a68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:19.749155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf0796-3f77-413f-aae8-ed3843122a68 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:19.757783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 777/1562] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:19.758635 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d4f67-f137-4309-ad11-af75c1c6a32c None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:19.761626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 786/1563] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:33:19.763471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6118a3-f58c-43a2-94a7-e7ffbf0335a3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.763768 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6118a3-f58c-43a2-94a7-e7ffbf0335a3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.764051 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6118a3-f58c-43a2-94a7-e7ffbf0335a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.768483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6d4f67-f137-4309-ad11-af75c1c6a32c None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:19.773149 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656b26e-05cf-417d-aa5e-d2e01a8824d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:19.773815 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656b26e-05cf-417d-aa5e-d2e01a8824d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.774104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656b26e-05cf-417d-aa5e-d2e01a8824d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.774387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656b26e-05cf-417d-aa5e-d2e01a8824d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.775123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656b26e-05cf-417d-aa5e-d2e01a8824d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:19.776807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656b26e-05cf-417d-aa5e-d2e01a8824d3 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:19.783112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1b6118a3-f58c-43a2-94a7-e7ffbf0335a3 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:19.784377 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1b6118a3-f58c-43a2-94a7-e7ffbf0335a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:19.787012 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 778/1564] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:19.787847 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656b26e-05cf-417d-aa5e-d2e01a8824d3 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:19.793428 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 787/1565] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:19.794272 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6118a3-f58c-43a2-94a7-e7ffbf0335a3 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:19.800672 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656b26e-05cf-417d-aa5e-d2e01a8824d3 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:19.803832 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6118a3-f58c-43a2-94a7-e7ffbf0335a3 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:19.805192 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a9d5d-c5e2-4236-b9f7-b76bc926fcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:19.805841 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a9d5d-c5e2-4236-b9f7-b76bc926fcda admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.806160 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a9d5d-c5e2-4236-b9f7-b76bc926fcda admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.806443 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6a9d5d-c5e2-4236-b9f7-b76bc926fcda admin admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.807165 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a9d5d-c5e2-4236-b9f7-b76bc926fcda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:33:19.808063 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a9d5d-c5e2-4236-b9f7-b76bc926fcda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:19.809003 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1858e-0d8d-4ae2-a5a0-0728dd6f80f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:19.809642 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1858e-0d8d-4ae2-a5a0-0728dd6f80f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.809770 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6a9d5d-c5e2-4236-b9f7-b76bc926fcda admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:19.809929 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1858e-0d8d-4ae2-a5a0-0728dd6f80f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.810207 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af1858e-0d8d-4ae2-a5a0-0728dd6f80f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.811063 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1858e-0d8d-4ae2-a5a0-0728dd6f80f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:33:19.811235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1858e-0d8d-4ae2-a5a0-0728dd6f80f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:19.812867 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af1858e-0d8d-4ae2-a5a0-0728dd6f80f1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:19.821587 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 779/1566] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:33:19.822386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a9d5d-c5e2-4236-b9f7-b76bc926fcda admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:19.831739 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 788/1567] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:33:19.831739 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1858e-0d8d-4ae2-a5a0-0728dd6f80f1 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:19.834101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6a9d5d-c5e2-4236-b9f7-b76bc926fcda admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:19.839341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284da073-36c3-46de-b6c6-0ff87ade7487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:19.840125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284da073-36c3-46de-b6c6-0ff87ade7487 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.840305 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284da073-36c3-46de-b6c6-0ff87ade7487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.840587 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284da073-36c3-46de-b6c6-0ff87ade7487 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.841318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284da073-36c3-46de-b6c6-0ff87ade7487 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:19.842559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af1858e-0d8d-4ae2-a5a0-0728dd6f80f1 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:19.842935 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284da073-36c3-46de-b6c6-0ff87ade7487 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:19.847246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8372e3-cc3f-495b-bcf1-9d1248aab334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:19.847882 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8372e3-cc3f-495b-bcf1-9d1248aab334 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.848173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8372e3-cc3f-495b-bcf1-9d1248aab334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.848454 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8372e3-cc3f-495b-bcf1-9d1248aab334 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.849147 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8372e3-cc3f-495b-bcf1-9d1248aab334 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:19.850161 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: WARNING py.warnings [None req-bf8372e3-cc3f-495b-bcf1-9d1248aab334 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 10:33:19.850161 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: warnings.warn(msg) Jun 30 10:33:19.850161 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: Jun 30 10:33:19.851396 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8372e3-cc3f-495b-bcf1-9d1248aab334 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:19.858733 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 780/1568] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:19.858733 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284da073-36c3-46de-b6c6-0ff87ade7487 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:19.864498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284da073-36c3-46de-b6c6-0ff87ade7487 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:19.869156 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2064ee-5502-4e19-b0cb-17401e3d0727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:19.870479 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2064ee-5502-4e19-b0cb-17401e3d0727 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.870882 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2064ee-5502-4e19-b0cb-17401e3d0727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.871165 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2064ee-5502-4e19-b0cb-17401e3d0727 admin admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.871928 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2064ee-5502-4e19-b0cb-17401e3d0727 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:33:19.872237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2064ee-5502-4e19-b0cb-17401e3d0727 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:19.873822 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2064ee-5502-4e19-b0cb-17401e3d0727 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:19.887028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 781/1569] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:33:19.887999 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2064ee-5502-4e19-b0cb-17401e3d0727 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:19.897331 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2064ee-5502-4e19-b0cb-17401e3d0727 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:19.901976 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedcba43-3f1e-4656-8e22-648e8d73ee7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:19.902600 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedcba43-3f1e-4656-8e22-648e8d73ee7e admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.902885 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedcba43-3f1e-4656-8e22-648e8d73ee7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.903162 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedcba43-3f1e-4656-8e22-648e8d73ee7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.903910 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedcba43-3f1e-4656-8e22-648e8d73ee7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:19.906051 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedcba43-3f1e-4656-8e22-648e8d73ee7e admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:19.947112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 789/1570] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:33:19 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:19.948053 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8372e3-cc3f-495b-bcf1-9d1248aab334 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:19.957772 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8372e3-cc3f-495b-bcf1-9d1248aab334 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:19.962300 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f05e6d-5ffe-428f-ba4a-8036a9a7961c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:19.962937 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f05e6d-5ffe-428f-ba4a-8036a9a7961c admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.963224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f05e6d-5ffe-428f-ba4a-8036a9a7961c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.963570 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f05e6d-5ffe-428f-ba4a-8036a9a7961c admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.964534 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f05e6d-5ffe-428f-ba4a-8036a9a7961c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:19.966131 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f05e6d-5ffe-428f-ba4a-8036a9a7961c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:19.976770 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 790/1571] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:19.977564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f05e6d-5ffe-428f-ba4a-8036a9a7961c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:19.987547 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f05e6d-5ffe-428f-ba4a-8036a9a7961c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:19.992110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968bc0c-35e9-4f7c-8624-3bff0fd9e400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:19.992735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968bc0c-35e9-4f7c-8624-3bff0fd9e400 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:19.993038 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968bc0c-35e9-4f7c-8624-3bff0fd9e400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:19.993302 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3968bc0c-35e9-4f7c-8624-3bff0fd9e400 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:19.994012 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3968bc0c-35e9-4f7c-8624-3bff0fd9e400 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:33:19.994318 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3968bc0c-35e9-4f7c-8624-3bff0fd9e400 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:19.996613 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3968bc0c-35e9-4f7c-8624-3bff0fd9e400 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:20.000868 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 782/1572] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:33:19 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:20.006515 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedcba43-3f1e-4656-8e22-648e8d73ee7e admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:20.008634 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 791/1573] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:33:20.009128 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968bc0c-35e9-4f7c-8624-3bff0fd9e400 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:20.018964 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3968bc0c-35e9-4f7c-8624-3bff0fd9e400 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:20.067075 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedcba43-3f1e-4656-8e22-648e8d73ee7e admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:20.071779 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f892d516-a9ce-45e7-98c2-23cf5521ec39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:20.072463 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892d516-a9ce-45e7-98c2-23cf5521ec39 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:20.072746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892d516-a9ce-45e7-98c2-23cf5521ec39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:20.073243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f892d516-a9ce-45e7-98c2-23cf5521ec39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:20.073965 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f892d516-a9ce-45e7-98c2-23cf5521ec39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:20.075902 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f892d516-a9ce-45e7-98c2-23cf5521ec39 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:20.076252 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9f17b-2097-49b7-9f97-7f28f297de15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:20.076915 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9f17b-2097-49b7-9f97-7f28f297de15 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:20.077199 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9f17b-2097-49b7-9f97-7f28f297de15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:20.077476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc9f17b-2097-49b7-9f97-7f28f297de15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:20.078163 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc9f17b-2097-49b7-9f97-7f28f297de15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:20.080116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc9f17b-2097-49b7-9f97-7f28f297de15 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:20.160727 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 783/1574] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:33:20 2022] POST /identity/v3/projects => generated 405 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:20.161610 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f892d516-a9ce-45e7-98c2-23cf5521ec39 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:20.192289 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:20.194460 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 792/1575] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:33:20 2022] POST /identity/v3/projects => generated 391 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:20.195233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9f17b-2097-49b7-9f97-7f28f297de15 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:20.208928 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:20.234643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f892d516-a9ce-45e7-98c2-23cf5521ec39 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:20.261822 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc9f17b-2097-49b7-9f97-7f28f297de15 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:20.266460 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f9821-6fa5-43ab-9fc2-a6a13f85600d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:20.269388 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f9821-6fa5-43ab-9fc2-a6a13f85600d admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:20.269388 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f9821-6fa5-43ab-9fc2-a6a13f85600d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:20.269388 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1f9821-6fa5-43ab-9fc2-a6a13f85600d admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:20.269388 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f9821-6fa5-43ab-9fc2-a6a13f85600d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:20.270327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1f9821-6fa5-43ab-9fc2-a6a13f85600d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:20.291555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f0f64-725d-4c4b-af1b-c8c7cec701b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:20.292774 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f0f64-725d-4c4b-af1b-c8c7cec701b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:20.293314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f0f64-725d-4c4b-af1b-c8c7cec701b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:20.293600 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f0f64-725d-4c4b-af1b-c8c7cec701b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:20.294332 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f0f64-725d-4c4b-af1b-c8c7cec701b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:20.296051 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f0f64-725d-4c4b-af1b-c8c7cec701b6 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:20.305773 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 784/1576] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:20.306719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f0f64-725d-4c4b-af1b-c8c7cec701b6 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:20.316436 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f0f64-725d-4c4b-af1b-c8c7cec701b6 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:20.321883 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e0c5e-e571-4257-b8ec-914626eec38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:20.322510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e0c5e-e571-4257-b8ec-914626eec38e admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:20.322797 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e0c5e-e571-4257-b8ec-914626eec38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:20.323189 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e0c5e-e571-4257-b8ec-914626eec38e admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:20.326958 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e0c5e-e571-4257-b8ec-914626eec38e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:20.326958 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e0c5e-e571-4257-b8ec-914626eec38e admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:20.334893 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 785/1577] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:20.335811 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e0c5e-e571-4257-b8ec-914626eec38e admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:20.345484 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e0c5e-e571-4257-b8ec-914626eec38e admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:20.349989 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee786d-87aa-4948-afe1-3f1805550272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:20.350843 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee786d-87aa-4948-afe1-3f1805550272 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:20.351127 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee786d-87aa-4948-afe1-3f1805550272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:20.351444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee786d-87aa-4948-afe1-3f1805550272 admin admin] PATH_INFO: `/v3/projects/8244c746798443a1a155ae40853e5e26/users/c7f8abef711c4cd2b53852b3a969b120/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:20.354786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 793/1578] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:33:20 2022] POST /identity/v3/projects => generated 373 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:20.356323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f9821-6fa5-43ab-9fc2-a6a13f85600d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:20.367959 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee786d-87aa-4948-afe1-3f1805550272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8244c746798443a1a155ae40853e5e26, user_id=c7f8abef711c4cd2b53852b3a969b120, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:20.369023 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: WARNING py.warnings [None req-60ee786d-87aa-4948-afe1-3f1805550272 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 10:33:20.369023 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: warnings.warn(msg) Jun 30 10:33:20.369023 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: Jun 30 10:33:20.371718 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ee786d-87aa-4948-afe1-3f1805550272 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:20.420695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1f9821-6fa5-43ab-9fc2-a6a13f85600d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:20.425313 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c137629-46ad-4cf2-a903-c908f2578ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:20.425979 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c137629-46ad-4cf2-a903-c908f2578ac2 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:20.426263 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c137629-46ad-4cf2-a903-c908f2578ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:20.426543 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c137629-46ad-4cf2-a903-c908f2578ac2 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:20.427229 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c137629-46ad-4cf2-a903-c908f2578ac2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:20.429208 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c137629-46ad-4cf2-a903-c908f2578ac2 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:20.478265 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 786/1579] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:20 2022] PUT /identity/v3/projects/8244c746798443a1a155ae40853e5e26/users/c7f8abef711c4cd2b53852b3a969b120/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:20.479240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee786d-87aa-4948-afe1-3f1805550272 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:20.536318 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 794/1580] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:33:20 2022] POST /identity/v3/users => generated 289 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:20.537417 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c137629-46ad-4cf2-a903-c908f2578ac2 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:20.547012 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c137629-46ad-4cf2-a903-c908f2578ac2 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:20.550884 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ee786d-87aa-4948-afe1-3f1805550272 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:20.551670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e93e6-0824-4754-895c-dcffc922cd52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:20.552328 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e93e6-0824-4754-895c-dcffc922cd52 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:20.552616 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e93e6-0824-4754-895c-dcffc922cd52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:20.552896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660e93e6-0824-4754-895c-dcffc922cd52 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:20.553586 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660e93e6-0824-4754-895c-dcffc922cd52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:20.555544 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660e93e6-0824-4754-895c-dcffc922cd52 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:20.555733 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6628c6c-535e-4ae5-8b01-77b2cabd50c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:20.556399 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6628c6c-535e-4ae5-8b01-77b2cabd50c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:20.556691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6628c6c-535e-4ae5-8b01-77b2cabd50c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:20.556975 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6628c6c-535e-4ae5-8b01-77b2cabd50c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:20.557671 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6628c6c-535e-4ae5-8b01-77b2cabd50c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:20.559749 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6628c6c-535e-4ae5-8b01-77b2cabd50c3 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:20.656603 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 795/1581] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:33:20 2022] POST /identity/v3/users => generated 303 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:20.657479 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e93e6-0824-4754-895c-dcffc922cd52 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:20.667754 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660e93e6-0824-4754-895c-dcffc922cd52 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:20.668582 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 787/1582] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:33:20 2022] POST /identity/v3/projects => generated 381 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:20.669240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6628c6c-535e-4ae5-8b01-77b2cabd50c3 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:20.686331 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:20.728285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a313aa-7a26-4331-b2b3-da22a6ae06dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:20.728963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a313aa-7a26-4331-b2b3-da22a6ae06dd admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:20.729351 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a313aa-7a26-4331-b2b3-da22a6ae06dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:20.729529 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a313aa-7a26-4331-b2b3-da22a6ae06dd admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:20.730221 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a313aa-7a26-4331-b2b3-da22a6ae06dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:20.732263 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a313aa-7a26-4331-b2b3-da22a6ae06dd admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:20.736453 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6628c6c-535e-4ae5-8b01-77b2cabd50c3 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:20.741685 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaaf8e0-468e-4bf4-8f30-4c1687d297d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:20.742156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:20.742614 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaaf8e0-468e-4bf4-8f30-4c1687d297d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:20.742898 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaaf8e0-468e-4bf4-8f30-4c1687d297d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:20.743436 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaaf8e0-468e-4bf4-8f30-4c1687d297d1 admin admin] PATH_INFO: `/v3/projects/c6c9c2c365864c2592431906f83c8e94/users/4cc6a79de538411ea88772cf5cc3bc1f/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:20.757897 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaaf8e0-468e-4bf4-8f30-4c1687d297d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6c9c2c365864c2592431906f83c8e94, user_id=4cc6a79de538411ea88772cf5cc3bc1f, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:20.761249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caaaf8e0-468e-4bf4-8f30-4c1687d297d1 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:20.809236 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 788/1583] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:20 2022] PUT /identity/v3/projects/c6c9c2c365864c2592431906f83c8e94/users/4cc6a79de538411ea88772cf5cc3bc1f/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:20.810158 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 796/1584] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:33:20 2022] POST /identity/v3/users => generated 296 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:20.810288 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaaf8e0-468e-4bf4-8f30-4c1687d297d1 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:20.810906 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a313aa-7a26-4331-b2b3-da22a6ae06dd admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:20.826728 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a313aa-7a26-4331-b2b3-da22a6ae06dd admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:20.877210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaaf8e0-468e-4bf4-8f30-4c1687d297d1 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:20.882823 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c641d506-61b1-40e5-a17f-0ab4b9b162c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:20.883665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641d506-61b1-40e5-a17f-0ab4b9b162c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:20.893253 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641d506-61b1-40e5-a17f-0ab4b9b162c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:20.893253 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c641d506-61b1-40e5-a17f-0ab4b9b162c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:20.893253 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641d506-61b1-40e5-a17f-0ab4b9b162c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:20.894432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23abcf-ebb4-45aa-8d66-4fb258d013b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:20.894432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23abcf-ebb4-45aa-8d66-4fb258d013b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:20.894432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23abcf-ebb4-45aa-8d66-4fb258d013b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:20.894432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b23abcf-ebb4-45aa-8d66-4fb258d013b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:20.894778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b23abcf-ebb4-45aa-8d66-4fb258d013b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:20.894863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c641d506-61b1-40e5-a17f-0ab4b9b162c0 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:20.894949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b23abcf-ebb4-45aa-8d66-4fb258d013b3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:20.901644 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 789/1585] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:20.901644 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7707ab-a625-48ab-9691-20563d23764b None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:20.901644 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7707ab-a625-48ab-9691-20563d23764b None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:20.901644 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7707ab-a625-48ab-9691-20563d23764b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:20.918088 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-ae7707ab-a625-48ab-9691-20563d23764b None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:20.925157 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-ae7707ab-a625-48ab-9691-20563d23764b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:20.926888 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 790/1586] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:20.927785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7707ab-a625-48ab-9691-20563d23764b None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:20.937541 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7707ab-a625-48ab-9691-20563d23764b None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:20.942110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634f445-4bf7-4108-935c-f78ba982d033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:20.942749 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634f445-4bf7-4108-935c-f78ba982d033 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:20.943030 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634f445-4bf7-4108-935c-f78ba982d033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:20.943327 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634f445-4bf7-4108-935c-f78ba982d033 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:20.944042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634f445-4bf7-4108-935c-f78ba982d033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:20.945717 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1634f445-4bf7-4108-935c-f78ba982d033 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:20.956001 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 791/1587] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:20.956762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634f445-4bf7-4108-935c-f78ba982d033 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:20.966535 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634f445-4bf7-4108-935c-f78ba982d033 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:20.971050 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b1ddfc-79d1-4f36-bc5a-dafa174def2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:20.971700 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1ddfc-79d1-4f36-bc5a-dafa174def2d admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:20.971986 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1ddfc-79d1-4f36-bc5a-dafa174def2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:20.972267 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1ddfc-79d1-4f36-bc5a-dafa174def2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:20.972963 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b1ddfc-79d1-4f36-bc5a-dafa174def2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:20.974517 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b1ddfc-79d1-4f36-bc5a-dafa174def2d admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:20.982238 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 797/1588] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:33:20 2022] POST /identity/v3/users => generated 287 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:20.983679 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23abcf-ebb4-45aa-8d66-4fb258d013b3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:20.988773 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 792/1589] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:20.988773 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc535a0-b3c8-4b03-8344-c7cf2f74d703 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:20.988773 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc535a0-b3c8-4b03-8344-c7cf2f74d703 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:20.988773 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc535a0-b3c8-4b03-8344-c7cf2f74d703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:20.994486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b23abcf-ebb4-45aa-8d66-4fb258d013b3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:21.000343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2b6ef3-57a5-4c60-b23e-ed73a2841047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:21.001009 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b6ef3-57a5-4c60-b23e-ed73a2841047 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:21.001290 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b6ef3-57a5-4c60-b23e-ed73a2841047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:21.001563 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b6ef3-57a5-4c60-b23e-ed73a2841047 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:21.002277 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2b6ef3-57a5-4c60-b23e-ed73a2841047 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:21.004277 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2b6ef3-57a5-4c60-b23e-ed73a2841047 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:21.008115 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-bbc535a0-b3c8-4b03-8344-c7cf2f74d703 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:21.008115 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-bbc535a0-b3c8-4b03-8344-c7cf2f74d703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:21.015496 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 793/1590] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:21.016272 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc535a0-b3c8-4b03-8344-c7cf2f74d703 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:21.025574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc535a0-b3c8-4b03-8344-c7cf2f74d703 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:21.030350 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c536c-0db3-43e3-b700-e9bf9a5c6eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:21.030996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c536c-0db3-43e3-b700-e9bf9a5c6eed admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:21.031297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c536c-0db3-43e3-b700-e9bf9a5c6eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:21.031576 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6c536c-0db3-43e3-b700-e9bf9a5c6eed admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:21.032296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c536c-0db3-43e3-b700-e9bf9a5c6eed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:21.033886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6c536c-0db3-43e3-b700-e9bf9a5c6eed admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:21.044698 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 794/1591] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:21.045445 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c536c-0db3-43e3-b700-e9bf9a5c6eed admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:21.054722 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6c536c-0db3-43e3-b700-e9bf9a5c6eed admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:21.059787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3c1e3-c9fe-4be7-885e-6b11d25a0de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:21.060614 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3c1e3-c9fe-4be7-885e-6b11d25a0de5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:21.065978 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3c1e3-c9fe-4be7-885e-6b11d25a0de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:21.065978 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3c1e3-c9fe-4be7-885e-6b11d25a0de5 admin admin] PATH_INFO: `/v3/projects/c8ee4130595948df9d462a6ca9f49c10/users/797a7b1e128f4102bac6430a60d2c0dc/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:21.077173 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3c1e3-c9fe-4be7-885e-6b11d25a0de5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8ee4130595948df9d462a6ca9f49c10, user_id=797a7b1e128f4102bac6430a60d2c0dc, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:21.079921 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3c1e3-c9fe-4be7-885e-6b11d25a0de5 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:21.095705 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 798/1592] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:33:20 2022] POST /identity/v3/users => generated 291 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:21.096597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2b6ef3-57a5-4c60-b23e-ed73a2841047 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:21.146899 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 795/1593] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:21 2022] PUT /identity/v3/projects/c8ee4130595948df9d462a6ca9f49c10/users/797a7b1e128f4102bac6430a60d2c0dc/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:21.147806 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3c1e3-c9fe-4be7-885e-6b11d25a0de5 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:21.150968 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2b6ef3-57a5-4c60-b23e-ed73a2841047 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:21.172958 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:21.202143 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706143bd-3353-41ad-b2da-3fb0b84c88da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:21.202704 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706143bd-3353-41ad-b2da-3fb0b84c88da service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:21.202985 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706143bd-3353-41ad-b2da-3fb0b84c88da service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:21.203255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706143bd-3353-41ad-b2da-3fb0b84c88da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:21.210136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3c1e3-c9fe-4be7-885e-6b11d25a0de5 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:21.212624 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706143bd-3353-41ad-b2da-3fb0b84c88da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:21.214292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706143bd-3353-41ad-b2da-3fb0b84c88da service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:21.215228 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f1149-ce36-4792-baab-c81006e35200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:21.215921 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f1149-ce36-4792-baab-c81006e35200 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:21.216209 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f1149-ce36-4792-baab-c81006e35200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:21.216489 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f1149-ce36-4792-baab-c81006e35200 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:21.217219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f1149-ce36-4792-baab-c81006e35200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:21.218810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50f1149-ce36-4792-baab-c81006e35200 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:21.224263 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 799/1594] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:21.224878 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706143bd-3353-41ad-b2da-3fb0b84c88da service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:21.229774 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 796/1595] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:21.231378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f1149-ce36-4792-baab-c81006e35200 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:21.234509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706143bd-3353-41ad-b2da-3fb0b84c88da service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:21.239446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23eb34b-af33-491e-b75d-dd83b7229588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:21.240180 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23eb34b-af33-491e-b75d-dd83b7229588 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:21.240443 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f1149-ce36-4792-baab-c81006e35200 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:21.240597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23eb34b-af33-491e-b75d-dd83b7229588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:21.240941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23eb34b-af33-491e-b75d-dd83b7229588 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:21.241744 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23eb34b-af33-491e-b75d-dd83b7229588 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:21.243422 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23eb34b-af33-491e-b75d-dd83b7229588 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:21.245916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4108598-8ccd-418f-a8a7-7cd98ae35905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:21.246927 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4108598-8ccd-418f-a8a7-7cd98ae35905 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:21.247214 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4108598-8ccd-418f-a8a7-7cd98ae35905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:21.247781 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4108598-8ccd-418f-a8a7-7cd98ae35905 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:21.248263 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4108598-8ccd-418f-a8a7-7cd98ae35905 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:21.250312 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4108598-8ccd-418f-a8a7-7cd98ae35905 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:21.254503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 800/1596] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:21.255428 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23eb34b-af33-491e-b75d-dd83b7229588 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:21.263429 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 797/1597] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:21.264277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4108598-8ccd-418f-a8a7-7cd98ae35905 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:21.269947 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23eb34b-af33-491e-b75d-dd83b7229588 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:21.274227 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d14b16-3eb1-4d10-8b7b-0c29784225eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:21.274395 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4108598-8ccd-418f-a8a7-7cd98ae35905 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:21.274959 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d14b16-3eb1-4d10-8b7b-0c29784225eb service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:21.275240 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d14b16-3eb1-4d10-8b7b-0c29784225eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:21.275541 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d14b16-3eb1-4d10-8b7b-0c29784225eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:21.279880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62247cb-643a-416d-a718-609d90fc8cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:21.280529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62247cb-643a-416d-a718-609d90fc8cc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:21.280814 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62247cb-643a-416d-a718-609d90fc8cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:21.281092 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62247cb-643a-416d-a718-609d90fc8cc9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:21.281816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62247cb-643a-416d-a718-609d90fc8cc9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:21.283437 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62247cb-643a-416d-a718-609d90fc8cc9 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:21.286097 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d14b16-3eb1-4d10-8b7b-0c29784225eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:21.287576 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d14b16-3eb1-4d10-8b7b-0c29784225eb service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:21.293448 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 798/1598] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:21.294076 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62247cb-643a-416d-a718-609d90fc8cc9 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:21.299088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 801/1599] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:21.303045 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87cb61-c4ba-4f94-880b-93254b313904 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:21.303626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87cb61-c4ba-4f94-880b-93254b313904 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:21.303895 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62247cb-643a-416d-a718-609d90fc8cc9 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:21.304033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87cb61-c4ba-4f94-880b-93254b313904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:21.308691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee50b61-2cac-4db3-8ffc-e76d27ccc3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:21.309504 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee50b61-2cac-4db3-8ffc-e76d27ccc3b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:21.309795 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee50b61-2cac-4db3-8ffc-e76d27ccc3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:21.310071 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee50b61-2cac-4db3-8ffc-e76d27ccc3b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:21.310813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee50b61-2cac-4db3-8ffc-e76d27ccc3b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:21.311058 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:21.312442 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee50b61-2cac-4db3-8ffc-e76d27ccc3b7 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:21.324038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 799/1600] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:21.325125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee50b61-2cac-4db3-8ffc-e76d27ccc3b7 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:21.326672 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5c87cb61-c4ba-4f94-880b-93254b313904 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:21.328174 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5c87cb61-c4ba-4f94-880b-93254b313904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:21.336411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 802/1601] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:21.336885 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee50b61-2cac-4db3-8ffc-e76d27ccc3b7 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:21.337997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87cb61-c4ba-4f94-880b-93254b313904 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:21.349798 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87cb61-c4ba-4f94-880b-93254b313904 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:21.351610 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bb717-79e4-4a37-9444-1a6d8e53bfb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:21.358881 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bb717-79e4-4a37-9444-1a6d8e53bfb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:21.358881 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bb717-79e4-4a37-9444-1a6d8e53bfb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:21.358881 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bb717-79e4-4a37-9444-1a6d8e53bfb0 admin admin] PATH_INFO: `/v3/projects/c96cea5375d54cb0aa93cdc2f24f150a/users/9aa86b4d8a924a068f6b4da6dbf5c6bd/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:21.359215 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008df00b-2e38-4ce8-a64d-9ba0d1931f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:21.359215 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008df00b-2e38-4ce8-a64d-9ba0d1931f65 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:21.359215 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008df00b-2e38-4ce8-a64d-9ba0d1931f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:21.359215 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008df00b-2e38-4ce8-a64d-9ba0d1931f65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:21.359594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008df00b-2e38-4ce8-a64d-9ba0d1931f65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:21.359594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008df00b-2e38-4ce8-a64d-9ba0d1931f65 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:21.370915 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 803/1602] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:21.371705 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008df00b-2e38-4ce8-a64d-9ba0d1931f65 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:21.379882 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1bb717-79e4-4a37-9444-1a6d8e53bfb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c96cea5375d54cb0aa93cdc2f24f150a, user_id=9aa86b4d8a924a068f6b4da6dbf5c6bd, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:21.384035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1bb717-79e4-4a37-9444-1a6d8e53bfb0 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:21.386462 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008df00b-2e38-4ce8-a64d-9ba0d1931f65 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:21.393764 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bec751-19d4-4a7a-89e1-eb793178805c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:21.393764 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bec751-19d4-4a7a-89e1-eb793178805c admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:21.393764 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bec751-19d4-4a7a-89e1-eb793178805c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:21.393764 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bec751-19d4-4a7a-89e1-eb793178805c admin admin] PATH_INFO: `/v3/projects/8ea8b7a5063c4f36b8df8fd5024259eb/users/624e8e4513b449cfaf0343e614685790/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:21.409159 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bec751-19d4-4a7a-89e1-eb793178805c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ea8b7a5063c4f36b8df8fd5024259eb, user_id=624e8e4513b449cfaf0343e614685790, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:21.411941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bec751-19d4-4a7a-89e1-eb793178805c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:21.508356 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 804/1603] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:21 2022] PUT /identity/v3/projects/8ea8b7a5063c4f36b8df8fd5024259eb/users/624e8e4513b449cfaf0343e614685790/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:21.509298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bec751-19d4-4a7a-89e1-eb793178805c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:21.519990 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 800/1604] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:21 2022] PUT /identity/v3/projects/c96cea5375d54cb0aa93cdc2f24f150a/users/9aa86b4d8a924a068f6b4da6dbf5c6bd/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:21.521365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bb717-79e4-4a37-9444-1a6d8e53bfb0 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:21.587710 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bec751-19d4-4a7a-89e1-eb793178805c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:21.593042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bb717-79e4-4a37-9444-1a6d8e53bfb0 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:21.593452 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0e1a2-e944-4857-a490-663525ab5271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:21.594465 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0e1a2-e944-4857-a490-663525ab5271 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:21.594929 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0e1a2-e944-4857-a490-663525ab5271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:21.595151 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda0e1a2-e944-4857-a490-663525ab5271 admin admin] PATH_INFO: `/v3/projects/c1955ca75a3b42d28474c3d7df74c9a6/users/eecc2db7b5754624b678e51dd9a37a46/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:21.597922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d91cb6-52b4-4821-8de9-cc917bfff468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:21.598606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d91cb6-52b4-4821-8de9-cc917bfff468 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:21.599009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d91cb6-52b4-4821-8de9-cc917bfff468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:21.599382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d91cb6-52b4-4821-8de9-cc917bfff468 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:21.600161 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d91cb6-52b4-4821-8de9-cc917bfff468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:21.601893 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d91cb6-52b4-4821-8de9-cc917bfff468 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:21.612878 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda0e1a2-e944-4857-a490-663525ab5271 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1955ca75a3b42d28474c3d7df74c9a6, user_id=eecc2db7b5754624b678e51dd9a37a46, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:21.613902 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 801/1605] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:21.614777 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d91cb6-52b4-4821-8de9-cc917bfff468 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:21.616673 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-95d91cb6-52b4-4821-8de9-cc917bfff468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:21.618384 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda0e1a2-e944-4857-a490-663525ab5271 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:21.677769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d91cb6-52b4-4821-8de9-cc917bfff468 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:21.689005 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 805/1606] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:21 2022] PUT /identity/v3/projects/c1955ca75a3b42d28474c3d7df74c9a6/users/eecc2db7b5754624b678e51dd9a37a46/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:21.689005 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0e1a2-e944-4857-a490-663525ab5271 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:21.691214 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:21.734781 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d60958-883d-4ba3-92a6-5d1669e4d459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:21.742148 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d60958-883d-4ba3-92a6-5d1669e4d459 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:21.742148 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d60958-883d-4ba3-92a6-5d1669e4d459 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:21.742148 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d60958-883d-4ba3-92a6-5d1669e4d459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:21.758676 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda0e1a2-e944-4857-a490-663525ab5271 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:21.764790 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f9d1d4-e5e9-4a17-9ba0-3a738ded7526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:21.772616 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9d1d4-e5e9-4a17-9ba0-3a738ded7526 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:21.772616 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9d1d4-e5e9-4a17-9ba0-3a738ded7526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:21.772616 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f9d1d4-e5e9-4a17-9ba0-3a738ded7526 admin admin] PATH_INFO: `/v3/projects/4c9afcac19364491989d502823267a41/users/022543d7d45f4708b39587f8d79bff94/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:21.782992 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f9d1d4-e5e9-4a17-9ba0-3a738ded7526 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c9afcac19364491989d502823267a41, user_id=022543d7d45f4708b39587f8d79bff94, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:21.785848 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f9d1d4-e5e9-4a17-9ba0-3a738ded7526 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:21.804136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d60958-883d-4ba3-92a6-5d1669e4d459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:21.805602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d60958-883d-4ba3-92a6-5d1669e4d459 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:21.816341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 802/1607] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:21.817654 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2b08d-5acb-4951-a393-84051ff8b8f5 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:21.817952 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2b08d-5acb-4951-a393-84051ff8b8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:21.821117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2b08d-5acb-4951-a393-84051ff8b8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:21.864579 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-26d2b08d-5acb-4951-a393-84051ff8b8f5 None None] MFA Rules not processed for user `624e8e4513b449cfaf0343e614685790`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:21.864579 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-26d2b08d-5acb-4951-a393-84051ff8b8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:21.871317 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 806/1608] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:21 2022] PUT /identity/v3/projects/4c9afcac19364491989d502823267a41/users/022543d7d45f4708b39587f8d79bff94/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:21.872570 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb5af5-af52-4971-bf61-04caf881446e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:21.872864 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb5af5-af52-4971-bf61-04caf881446e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:21.873208 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb5af5-af52-4971-bf61-04caf881446e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:21.898853 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-77eb5af5-af52-4971-bf61-04caf881446e None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:21.898853 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-77eb5af5-af52-4971-bf61-04caf881446e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:21.924847 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:21.942588 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 803/1609] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:21 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:21.943518 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2b08d-5acb-4951-a393-84051ff8b8f5 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:21.991444 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 807/1610] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:21.995150 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3e7fb-71eb-442b-b3ef-4b8a318d8058 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:21.995577 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3e7fb-71eb-442b-b3ef-4b8a318d8058 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:21.996547 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c3e7fb-71eb-442b-b3ef-4b8a318d8058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:22.042600 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d7c3e7fb-71eb-442b-b3ef-4b8a318d8058 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:22.048584 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d7c3e7fb-71eb-442b-b3ef-4b8a318d8058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:22.056382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:22.069884 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2b08d-5acb-4951-a393-84051ff8b8f5 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:22.071862 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 808/1611] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:22.077087 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c3c51-b28f-49b2-9383-77c576d4be50 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:22.081542 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3484145-33c6-41e8-8d3f-c9e6b1c215e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:22.081542 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3484145-33c6-41e8-8d3f-c9e6b1c215e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:22.081969 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c3c51-b28f-49b2-9383-77c576d4be50 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:22.082077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3484145-33c6-41e8-8d3f-c9e6b1c215e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:22.082195 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c3c51-b28f-49b2-9383-77c576d4be50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:22.082292 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3484145-33c6-41e8-8d3f-c9e6b1c215e1 admin admin] PATH_INFO: `/v3/projects/f492cb5130db4570bd2a4445fa3128e6/users/30ffcd735910410fa38e6d24e19fd7e2/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:22.100627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3484145-33c6-41e8-8d3f-c9e6b1c215e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f492cb5130db4570bd2a4445fa3128e6, user_id=30ffcd735910410fa38e6d24e19fd7e2, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:22.103966 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3484145-33c6-41e8-8d3f-c9e6b1c215e1 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:22.112945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b68c3c51-b28f-49b2-9383-77c576d4be50 None None] MFA Rules not processed for user `022543d7d45f4708b39587f8d79bff94`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:22.114174 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b68c3c51-b28f-49b2-9383-77c576d4be50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:22.176832 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 804/1612] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:21 2022] PUT /identity/v3/projects/f492cb5130db4570bd2a4445fa3128e6/users/30ffcd735910410fa38e6d24e19fd7e2/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:22.177558 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3484145-33c6-41e8-8d3f-c9e6b1c215e1 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:22.188500 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 809/1613] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:22.189183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c3c51-b28f-49b2-9383-77c576d4be50 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:22.264093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c3c51-b28f-49b2-9383-77c576d4be50 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:22.269237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3484145-33c6-41e8-8d3f-c9e6b1c215e1 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:22.277457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4d9a75-0d8b-4a88-b6af-e0dceb42c5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:22.278528 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d9a75-0d8b-4a88-b6af-e0dceb42c5fb admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:22.278754 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d9a75-0d8b-4a88-b6af-e0dceb42c5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:22.279303 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4d9a75-0d8b-4a88-b6af-e0dceb42c5fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:22.280627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d9a75-0d8b-4a88-b6af-e0dceb42c5fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:22.282776 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4d9a75-0d8b-4a88-b6af-e0dceb42c5fb admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:22.333929 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99587b71-3f2d-4001-9401-8d6ba6fc6e5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:22.335539 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99587b71-3f2d-4001-9401-8d6ba6fc6e5b service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:22.336439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99587b71-3f2d-4001-9401-8d6ba6fc6e5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:22.337015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99587b71-3f2d-4001-9401-8d6ba6fc6e5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:22.350114 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99587b71-3f2d-4001-9401-8d6ba6fc6e5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:22.351970 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99587b71-3f2d-4001-9401-8d6ba6fc6e5b service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:22.362861 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 810/1614] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:22.363577 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99587b71-3f2d-4001-9401-8d6ba6fc6e5b service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:22.372695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99587b71-3f2d-4001-9401-8d6ba6fc6e5b service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:22.376760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6fff34-8bdd-4b24-9a22-a71530799506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:22.377410 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fff34-8bdd-4b24-9a22-a71530799506 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:22.377823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fff34-8bdd-4b24-9a22-a71530799506 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:22.377943 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6fff34-8bdd-4b24-9a22-a71530799506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:22.390040 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 805/1615] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:33:22 2022] POST /identity/v3/projects => generated 405 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:22.394586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cbb7f-a0ba-4e1d-829f-45b32b66e318 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:22.394879 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cbb7f-a0ba-4e1d-829f-45b32b66e318 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:22.395177 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cbb7f-a0ba-4e1d-829f-45b32b66e318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:22.403360 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:22.407718 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6fff34-8bdd-4b24-9a22-a71530799506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:22.412978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6fff34-8bdd-4b24-9a22-a71530799506 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:22.426243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-2f7cbb7f-a0ba-4e1d-829f-45b32b66e318 None None] MFA Rules not processed for user `30ffcd735910410fa38e6d24e19fd7e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:22.427743 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-2f7cbb7f-a0ba-4e1d-829f-45b32b66e318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:22.428713 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 811/1616] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:22.429748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6fff34-8bdd-4b24-9a22-a71530799506 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:22.447009 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6fff34-8bdd-4b24-9a22-a71530799506 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:22.523886 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415bc90-c5a0-49d5-ba3c-5c1c07235956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:22.524835 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415bc90-c5a0-49d5-ba3c-5c1c07235956 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:22.524835 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415bc90-c5a0-49d5-ba3c-5c1c07235956 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:22.525573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5415bc90-c5a0-49d5-ba3c-5c1c07235956 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:22.525910 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5415bc90-c5a0-49d5-ba3c-5c1c07235956 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:22.528679 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5415bc90-c5a0-49d5-ba3c-5c1c07235956 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:22.532263 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 806/1617] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:22.534529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cbb7f-a0ba-4e1d-829f-45b32b66e318 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:22.553489 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cbb7f-a0ba-4e1d-829f-45b32b66e318 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:22.561817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1049cff-f8cc-4c91-97ab-c610797899cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:22.563665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1049cff-f8cc-4c91-97ab-c610797899cb admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:22.564022 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1049cff-f8cc-4c91-97ab-c610797899cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:22.564917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1049cff-f8cc-4c91-97ab-c610797899cb admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:22.565529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1049cff-f8cc-4c91-97ab-c610797899cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:22.570838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1049cff-f8cc-4c91-97ab-c610797899cb admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:22.593255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 812/1618] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:33:22 2022] POST /identity/v3/projects => generated 371 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:22.594698 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415bc90-c5a0-49d5-ba3c-5c1c07235956 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:22.664612 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 807/1619] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:33:22 2022] POST /identity/v3/users => generated 309 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:22.665873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1049cff-f8cc-4c91-97ab-c610797899cb admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:22.679366 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1049cff-f8cc-4c91-97ab-c610797899cb admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:22.683308 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5415bc90-c5a0-49d5-ba3c-5c1c07235956 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:22.687710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8238563-6feb-4e6c-9926-a2fcc3acab72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:22.691846 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8238563-6feb-4e6c-9926-a2fcc3acab72 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:22.691846 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8238563-6feb-4e6c-9926-a2fcc3acab72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:22.691846 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8238563-6feb-4e6c-9926-a2fcc3acab72 admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:22.691846 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8238563-6feb-4e6c-9926-a2fcc3acab72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:22.692263 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf2880-2987-4260-9946-5ff72eca1d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:22.692263 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf2880-2987-4260-9946-5ff72eca1d33 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:22.692263 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf2880-2987-4260-9946-5ff72eca1d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:22.692263 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaf2880-2987-4260-9946-5ff72eca1d33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:22.692622 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8238563-6feb-4e6c-9926-a2fcc3acab72 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:22.694035 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf2880-2987-4260-9946-5ff72eca1d33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:22.696846 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aaf2880-2987-4260-9946-5ff72eca1d33 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:22.781013 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 808/1620] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:33:22 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:22.784918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8238563-6feb-4e6c-9926-a2fcc3acab72 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:22.795663 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8238563-6feb-4e6c-9926-a2fcc3acab72 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:22.800764 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfff285-e8d5-40fd-a693-bcba471ad278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:22.806567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfff285-e8d5-40fd-a693-bcba471ad278 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:22.806567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfff285-e8d5-40fd-a693-bcba471ad278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:22.806567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfff285-e8d5-40fd-a693-bcba471ad278 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:22.806567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfff285-e8d5-40fd-a693-bcba471ad278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:22.806567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfff285-e8d5-40fd-a693-bcba471ad278 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:22.808646 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 813/1621] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:33:22 2022] POST /identity/v3/projects => generated 383 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:22.809674 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf2880-2987-4260-9946-5ff72eca1d33 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:22.824731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 809/1622] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:22.825601 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfff285-e8d5-40fd-a693-bcba471ad278 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:22.832512 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:22.839469 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfff285-e8d5-40fd-a693-bcba471ad278 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:22.878738 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaf2880-2987-4260-9946-5ff72eca1d33 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:22.886988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0630d2-1ac9-444d-a278-0c9fabd19c46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:22.886988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0630d2-1ac9-444d-a278-0c9fabd19c46 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:22.886988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0630d2-1ac9-444d-a278-0c9fabd19c46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:22.886988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0630d2-1ac9-444d-a278-0c9fabd19c46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:22.887398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0630d2-1ac9-444d-a278-0c9fabd19c46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:22.888395 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0630d2-1ac9-444d-a278-0c9fabd19c46 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:22.904555 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 814/1623] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:22.905404 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0630d2-1ac9-444d-a278-0c9fabd19c46 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:22.913033 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26604a-d9d1-41de-9253-9c47a8c17e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:22.913785 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:22.923770 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26604a-d9d1-41de-9253-9c47a8c17e8e admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:22.923770 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26604a-d9d1-41de-9253-9c47a8c17e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:22.923770 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e26604a-d9d1-41de-9253-9c47a8c17e8e admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:22.923770 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e26604a-d9d1-41de-9253-9c47a8c17e8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:22.923770 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e26604a-d9d1-41de-9253-9c47a8c17e8e admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:22.924365 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0630d2-1ac9-444d-a278-0c9fabd19c46 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:22.927158 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87470e47-778c-407a-9bce-25dccb49d8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:22.928417 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87470e47-778c-407a-9bce-25dccb49d8b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:22.928702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87470e47-778c-407a-9bce-25dccb49d8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:22.928978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87470e47-778c-407a-9bce-25dccb49d8b1 admin admin] PATH_INFO: `/v3/projects/3fa369cb54eb481b8b97593178e6daa5/users/8812e4c2be15460ba28f4d4f17e1ee43/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:22.944823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87470e47-778c-407a-9bce-25dccb49d8b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fa369cb54eb481b8b97593178e6daa5, user_id=8812e4c2be15460ba28f4d4f17e1ee43, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:22.947683 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87470e47-778c-407a-9bce-25dccb49d8b1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:23.037787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 810/1624] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:33:22 2022] POST /identity/v3/users => generated 298 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:23.037787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26604a-d9d1-41de-9253-9c47a8c17e8e admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:23.052771 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e26604a-d9d1-41de-9253-9c47a8c17e8e admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:23.059321 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 815/1625] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:22 2022] PUT /identity/v3/projects/3fa369cb54eb481b8b97593178e6daa5/users/8812e4c2be15460ba28f4d4f17e1ee43/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:23.060805 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87470e47-778c-407a-9bce-25dccb49d8b1 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:23.147955 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5828f-a05e-4176-9e8c-d3620e14e62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:23.149096 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5828f-a05e-4176-9e8c-d3620e14e62c admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:23.149847 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5828f-a05e-4176-9e8c-d3620e14e62c admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:23.150410 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5828f-a05e-4176-9e8c-d3620e14e62c admin admin] PATH_INFO: `/v3/projects/bca1d43679df47768d3bbc39f584f497/users/20757c9be9ad4b0aab725049b6a6f4e9/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:23.162622 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87470e47-778c-407a-9bce-25dccb49d8b1 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:23.170125 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7986f-2eda-4ff1-a169-b9a86b5aed28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:23.171202 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7986f-2eda-4ff1-a169-b9a86b5aed28 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:23.175427 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7986f-2eda-4ff1-a169-b9a86b5aed28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:23.175427 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7986f-2eda-4ff1-a169-b9a86b5aed28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:23.175427 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7986f-2eda-4ff1-a169-b9a86b5aed28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:23.175427 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7986f-2eda-4ff1-a169-b9a86b5aed28 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:23.187538 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 816/1626] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:23.188984 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72152a-a094-46cd-a01d-d3815b78a0fb None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:23.189273 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72152a-a094-46cd-a01d-d3815b78a0fb None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:23.189790 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb72152a-a094-46cd-a01d-d3815b78a0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:23.192772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5828f-a05e-4176-9e8c-d3620e14e62c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bca1d43679df47768d3bbc39f584f497, user_id=20757c9be9ad4b0aab725049b6a6f4e9, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:23.200425 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5828f-a05e-4176-9e8c-d3620e14e62c admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:23.224260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-eb72152a-a094-46cd-a01d-d3815b78a0fb None None] MFA Rules not processed for user `4cc6a79de538411ea88772cf5cc3bc1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:23.225600 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-eb72152a-a094-46cd-a01d-d3815b78a0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:23.302341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 811/1627] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:23 2022] PUT /identity/v3/projects/bca1d43679df47768d3bbc39f584f497/users/20757c9be9ad4b0aab725049b6a6f4e9/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:23.303691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5828f-a05e-4176-9e8c-d3620e14e62c admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:23.317159 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 817/1628] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:23 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:23.318292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72152a-a094-46cd-a01d-d3815b78a0fb None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:23.348549 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb72152a-a094-46cd-a01d-d3815b78a0fb None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:23.411054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5828f-a05e-4176-9e8c-d3620e14e62c admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:23.413633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a6f77-42a2-48ab-9e6a-6e72083313f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:23.414506 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a6f77-42a2-48ab-9e6a-6e72083313f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:23.414921 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a6f77-42a2-48ab-9e6a-6e72083313f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:23.415421 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73a6f77-42a2-48ab-9e6a-6e72083313f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:23.418925 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73a6f77-42a2-48ab-9e6a-6e72083313f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:23.423762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73a6f77-42a2-48ab-9e6a-6e72083313f3 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:23.447343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 812/1629] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:23.449093 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a6f77-42a2-48ab-9e6a-6e72083313f3 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:23.465937 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73a6f77-42a2-48ab-9e6a-6e72083313f3 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:23.482345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b41a1-b324-42d3-99ea-49cc0a32d64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:23.490010 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda510a-695e-4fb6-9ef9-c17ca00c5b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:23.490581 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda510a-695e-4fb6-9ef9-c17ca00c5b60 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:23.490884 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda510a-695e-4fb6-9ef9-c17ca00c5b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:23.491192 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcda510a-695e-4fb6-9ef9-c17ca00c5b60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:23.491405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b41a1-b324-42d3-99ea-49cc0a32d64d admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:23.492106 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b41a1-b324-42d3-99ea-49cc0a32d64d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:23.492253 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcda510a-695e-4fb6-9ef9-c17ca00c5b60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:23.492350 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b41a1-b324-42d3-99ea-49cc0a32d64d admin admin] PATH_INFO: `/v3/projects/7547a6aa33724b48a45578255410924c/users/27acf141898f45afb4b692dcba09b56f/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:23.493813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcda510a-695e-4fb6-9ef9-c17ca00c5b60 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:23.509023 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 813/1630] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:23.509989 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda510a-695e-4fb6-9ef9-c17ca00c5b60 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:23.517974 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:23.521284 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:23.526596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcda510a-695e-4fb6-9ef9-c17ca00c5b60 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:23.529760 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a6b8a-b8b2-49be-8164-911067a0a079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:23.530461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a6b8a-b8b2-49be-8164-911067a0a079 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:23.530753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a6b8a-b8b2-49be-8164-911067a0a079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:23.531044 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3a6b8a-b8b2-49be-8164-911067a0a079 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:23.531849 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3a6b8a-b8b2-49be-8164-911067a0a079 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:23.533873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3a6b8a-b8b2-49be-8164-911067a0a079 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:23.630908 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 814/1631] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:33:23 2022] POST /identity/v3/projects => generated 403 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:23.632177 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a6b8a-b8b2-49be-8164-911067a0a079 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:23.689898 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b41a1-b324-42d3-99ea-49cc0a32d64d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7547a6aa33724b48a45578255410924c, user_id=27acf141898f45afb4b692dcba09b56f, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:23.694163 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755b41a1-b324-42d3-99ea-49cc0a32d64d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:23.735543 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3a6b8a-b8b2-49be-8164-911067a0a079 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:23.742452 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c61db-879d-4f7e-8248-3c0da47b6c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:23.744105 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c61db-879d-4f7e-8248-3c0da47b6c84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:23.744697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c61db-879d-4f7e-8248-3c0da47b6c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:23.746390 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093c61db-879d-4f7e-8248-3c0da47b6c84 admin admin] PATH_INFO: `/v3/projects/3fa369cb54eb481b8b97593178e6daa5/users/8812e4c2be15460ba28f4d4f17e1ee43/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:23.749279 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093c61db-879d-4f7e-8248-3c0da47b6c84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fa369cb54eb481b8b97593178e6daa5, user_id=8812e4c2be15460ba28f4d4f17e1ee43, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:23.752294 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093c61db-879d-4f7e-8248-3c0da47b6c84 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:23.764666 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 818/1632] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:23 2022] PUT /identity/v3/projects/7547a6aa33724b48a45578255410924c/users/27acf141898f45afb4b692dcba09b56f/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:23.765848 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b41a1-b324-42d3-99ea-49cc0a32d64d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:23.788141 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 815/1633] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:23 2022] PUT /identity/v3/projects/3fa369cb54eb481b8b97593178e6daa5/users/8812e4c2be15460ba28f4d4f17e1ee43/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:23.797098 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c61db-879d-4f7e-8248-3c0da47b6c84 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:23.901623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093c61db-879d-4f7e-8248-3c0da47b6c84 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:23.909378 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755b41a1-b324-42d3-99ea-49cc0a32d64d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:23.914601 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f37500-f8c2-466d-a239-a30af3fe0de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:23.916144 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f37500-f8c2-466d-a239-a30af3fe0de6 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:23.919452 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f37500-f8c2-466d-a239-a30af3fe0de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:23.923547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f37500-f8c2-466d-a239-a30af3fe0de6 admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:23.923547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f37500-f8c2-466d-a239-a30af3fe0de6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:23.923762 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27e5ee-02b1-4c29-bd06-1a07df544749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:23.923762 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27e5ee-02b1-4c29-bd06-1a07df544749 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:23.923980 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f37500-f8c2-466d-a239-a30af3fe0de6 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:23.924076 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27e5ee-02b1-4c29-bd06-1a07df544749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:23.924076 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27e5ee-02b1-4c29-bd06-1a07df544749 admin admin] PATH_INFO: `/v3/projects/bca1d43679df47768d3bbc39f584f497/users/20757c9be9ad4b0aab725049b6a6f4e9/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:23.925803 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27e5ee-02b1-4c29-bd06-1a07df544749 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bca1d43679df47768d3bbc39f584f497, user_id=20757c9be9ad4b0aab725049b6a6f4e9, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:23.929577 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27e5ee-02b1-4c29-bd06-1a07df544749 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:23.957709 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 819/1634] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:23 2022] PUT /identity/v3/projects/bca1d43679df47768d3bbc39f584f497/users/20757c9be9ad4b0aab725049b6a6f4e9/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:23.958580 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27e5ee-02b1-4c29-bd06-1a07df544749 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:24.032330 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 816/1635] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:33:23 2022] POST /identity/v3/users => generated 308 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:24.034803 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da653bf2-972b-4e8e-a768-98a7f5689141 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:24.036448 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da653bf2-972b-4e8e-a768-98a7f5689141 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:24.036977 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da653bf2-972b-4e8e-a768-98a7f5689141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:24.058053 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27e5ee-02b1-4c29-bd06-1a07df544749 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:24.063783 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec6b54-5877-46c8-8efd-2bc294349fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:24.066511 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec6b54-5877-46c8-8efd-2bc294349fe0 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:24.066811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec6b54-5877-46c8-8efd-2bc294349fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:24.067090 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec6b54-5877-46c8-8efd-2bc294349fe0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:24.067917 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec6b54-5877-46c8-8efd-2bc294349fe0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:24.069682 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec6b54-5877-46c8-8efd-2bc294349fe0 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:24.085476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 820/1636] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:24.085476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d59f6-c04b-429f-9369-89caf645326e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:24.085476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d59f6-c04b-429f-9369-89caf645326e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:24.085476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4d59f6-c04b-429f-9369-89caf645326e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:24.107620 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-da653bf2-972b-4e8e-a768-98a7f5689141 None None] MFA Rules not processed for user `8812e4c2be15460ba28f4d4f17e1ee43`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:24.113252 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-da653bf2-972b-4e8e-a768-98a7f5689141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:24.116974 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8c4d59f6-c04b-429f-9369-89caf645326e None None] MFA Rules not processed for user `20757c9be9ad4b0aab725049b6a6f4e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:24.118400 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8c4d59f6-c04b-429f-9369-89caf645326e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:24.220075 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 821/1637] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:24 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:24.221347 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d59f6-c04b-429f-9369-89caf645326e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:24.243482 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:24.250706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:24.257029 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4d59f6-c04b-429f-9369-89caf645326e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:24.271600 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9922a3e-bd43-49e5-b6a9-ae8dc4430938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:24.273809 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9922a3e-bd43-49e5-b6a9-ae8dc4430938 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:24.273809 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9922a3e-bd43-49e5-b6a9-ae8dc4430938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:24.273809 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9922a3e-bd43-49e5-b6a9-ae8dc4430938 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:24.273809 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9922a3e-bd43-49e5-b6a9-ae8dc4430938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:24.277963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9922a3e-bd43-49e5-b6a9-ae8dc4430938 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:24.291586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 817/1638] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:24 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:24.293382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da653bf2-972b-4e8e-a768-98a7f5689141 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:24.309999 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da653bf2-972b-4e8e-a768-98a7f5689141 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:24.310483 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 822/1639] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:24 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:24.311605 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9922a3e-bd43-49e5-b6a9-ae8dc4430938 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:24.318544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170dd46-8d9c-465c-9c56-7007573cf573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:24.319798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170dd46-8d9c-465c-9c56-7007573cf573 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:24.325199 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170dd46-8d9c-465c-9c56-7007573cf573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:24.325199 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1170dd46-8d9c-465c-9c56-7007573cf573 admin admin] PATH_INFO: `/v3/projects/7547a6aa33724b48a45578255410924c/users/27acf141898f45afb4b692dcba09b56f/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:24.329732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1170dd46-8d9c-465c-9c56-7007573cf573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7547a6aa33724b48a45578255410924c, user_id=27acf141898f45afb4b692dcba09b56f, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:24.334288 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1170dd46-8d9c-465c-9c56-7007573cf573 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:24.354042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 818/1640] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:24 2022] PUT /identity/v3/projects/7547a6aa33724b48a45578255410924c/users/27acf141898f45afb4b692dcba09b56f/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:24.355680 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170dd46-8d9c-465c-9c56-7007573cf573 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:24.437564 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1170dd46-8d9c-465c-9c56-7007573cf573 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:24.442407 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177ba506-46b3-40dd-ad0b-2812731dc855 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:24.443073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ba506-46b3-40dd-ad0b-2812731dc855 service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:24.443509 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ba506-46b3-40dd-ad0b-2812731dc855 service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:24.443910 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177ba506-46b3-40dd-ad0b-2812731dc855 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:24.449630 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9922a3e-bd43-49e5-b6a9-ae8dc4430938 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:24.460851 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3974818-3f5d-4382-bc2c-20ce8640ceb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:24.461777 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3974818-3f5d-4382-bc2c-20ce8640ceb7 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:24.462225 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3974818-3f5d-4382-bc2c-20ce8640ceb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:24.463079 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3974818-3f5d-4382-bc2c-20ce8640ceb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:24.571362 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ba506-46b3-40dd-ad0b-2812731dc855 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:24.574535 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177ba506-46b3-40dd-ad0b-2812731dc855 service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:24.576352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3974818-3f5d-4382-bc2c-20ce8640ceb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:24.578260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3974818-3f5d-4382-bc2c-20ce8640ceb7 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:24.593386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 823/1641] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:24 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:24.594451 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3974818-3f5d-4382-bc2c-20ce8640ceb7 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:24.650762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 819/1642] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:24 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:24.654470 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd88f012-3a1c-4dc3-a41c-34873ab799a9 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:24.655031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd88f012-3a1c-4dc3-a41c-34873ab799a9 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:24.655555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd88f012-3a1c-4dc3-a41c-34873ab799a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:24.727499 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3974818-3f5d-4382-bc2c-20ce8640ceb7 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:24.744288 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9bb39-b9f9-4cfa-87e6-709216134915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:24.745410 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-bd88f012-3a1c-4dc3-a41c-34873ab799a9 None None] MFA Rules not processed for user `27acf141898f45afb4b692dcba09b56f`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:24.746991 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-bd88f012-3a1c-4dc3-a41c-34873ab799a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:24.747717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9bb39-b9f9-4cfa-87e6-709216134915 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:24.748022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9bb39-b9f9-4cfa-87e6-709216134915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:24.748631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9bb39-b9f9-4cfa-87e6-709216134915 admin admin] PATH_INFO: `/v3/projects/473aa1f80d2644c19b829e97f7fb60d1/users/fadae20ed280461e806b7b7d06c7b005/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:24.804769 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9bb39-b9f9-4cfa-87e6-709216134915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=473aa1f80d2644c19b829e97f7fb60d1, user_id=fadae20ed280461e806b7b7d06c7b005, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:24.808757 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba9bb39-b9f9-4cfa-87e6-709216134915 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:24.923316 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 820/1643] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:24.936264 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd88f012-3a1c-4dc3-a41c-34873ab799a9 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:24.951970 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd88f012-3a1c-4dc3-a41c-34873ab799a9 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:24.955037 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 824/1644] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:24 2022] PUT /identity/v3/projects/473aa1f80d2644c19b829e97f7fb60d1/users/fadae20ed280461e806b7b7d06c7b005/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:24.956308 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9bb39-b9f9-4cfa-87e6-709216134915 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:25.035699 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26017efb-650e-4799-965b-6b3d51adfbc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:25.035699 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26017efb-650e-4799-965b-6b3d51adfbc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:25.035699 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26017efb-650e-4799-965b-6b3d51adfbc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:25.035699 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26017efb-650e-4799-965b-6b3d51adfbc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:25.173831 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9bb39-b9f9-4cfa-87e6-709216134915 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:25.183726 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26017efb-650e-4799-965b-6b3d51adfbc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:25.186953 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26017efb-650e-4799-965b-6b3d51adfbc7 service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:25.193680 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:25.194574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde527e7-ca9d-4bfe-b14a-ffefc8498bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:25.195482 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde527e7-ca9d-4bfe-b14a-ffefc8498bf1 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:25.202090 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde527e7-ca9d-4bfe-b14a-ffefc8498bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:25.202090 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde527e7-ca9d-4bfe-b14a-ffefc8498bf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:25.202090 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde527e7-ca9d-4bfe-b14a-ffefc8498bf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:25.202090 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde527e7-ca9d-4bfe-b14a-ffefc8498bf1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:25.212267 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 821/1645] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:33:25.214028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26017efb-650e-4799-965b-6b3d51adfbc7 service cinder] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:25.221332 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 825/1646] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:24 2022] GET /identity/v3/roles => generated 1420 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:25.227291 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde527e7-ca9d-4bfe-b14a-ffefc8498bf1 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:25.244921 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26017efb-650e-4799-965b-6b3d51adfbc7 service cinder] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:25.257945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde527e7-ca9d-4bfe-b14a-ffefc8498bf1 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:25.258919 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc0fb5-3b2e-4bc1-bbbc-f1b1f430de69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:25.260973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc0fb5-3b2e-4bc1-bbbc-f1b1f430de69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:25.261806 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc0fb5-3b2e-4bc1-bbbc-f1b1f430de69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:25.262432 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bc0fb5-3b2e-4bc1-bbbc-f1b1f430de69 admin admin] PATH_INFO: `/v3/users/624e8e4513b449cfaf0343e614685790` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:25.264204 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe60fa6-6a8f-45bf-a58a-c923bb62f9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:25.265874 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe60fa6-6a8f-45bf-a58a-c923bb62f9c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:25.266169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe60fa6-6a8f-45bf-a58a-c923bb62f9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:25.266447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe60fa6-6a8f-45bf-a58a-c923bb62f9c2 admin admin] PATH_INFO: `/v3/projects/473aa1f80d2644c19b829e97f7fb60d1/users/fadae20ed280461e806b7b7d06c7b005/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:25.267044 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc0fb5-3b2e-4bc1-bbbc-f1b1f430de69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=624e8e4513b449cfaf0343e614685790)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:25.268557 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe60fa6-6a8f-45bf-a58a-c923bb62f9c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=473aa1f80d2644c19b829e97f7fb60d1, user_id=fadae20ed280461e806b7b7d06c7b005, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:25.271439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: WARNING py.warnings [None req-86bc0fb5-3b2e-4bc1-bbbc-f1b1f430de69 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 10:33:25.271439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: warnings.warn(msg) Jun 30 10:33:25.271439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: Jun 30 10:33:25.272881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe60fa6-6a8f-45bf-a58a-c923bb62f9c2 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:25.274005 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86bc0fb5-3b2e-4bc1-bbbc-f1b1f430de69 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:25.306485 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 826/1647] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:25 2022] PUT /identity/v3/projects/473aa1f80d2644c19b829e97f7fb60d1/users/fadae20ed280461e806b7b7d06c7b005/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:25.385609 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe60fa6-6a8f-45bf-a58a-c923bb62f9c2 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:25.644152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:25.648720 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe60fa6-6a8f-45bf-a58a-c923bb62f9c2 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:25.648720 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f874d179-4fc9-4f11-8ef8-5792caadeb54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:25.648720 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874d179-4fc9-4f11-8ef8-5792caadeb54 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:25.648720 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874d179-4fc9-4f11-8ef8-5792caadeb54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:25.652420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f874d179-4fc9-4f11-8ef8-5792caadeb54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:25.653889 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-86bc0fb5-3b2e-4bc1-bbbc-f1b1f430de69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '624e8e4513b449cfaf0343e614685790'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:25.665791 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f874d179-4fc9-4f11-8ef8-5792caadeb54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:25.671917 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f874d179-4fc9-4f11-8ef8-5792caadeb54 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:25.708957 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 827/1648] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:33:25.844543 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-86bc0fb5-3b2e-4bc1-bbbc-f1b1f430de69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '624e8e4513b449cfaf0343e614685790'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:25.891686 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-86bc0fb5-3b2e-4bc1-bbbc-f1b1f430de69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '624e8e4513b449cfaf0343e614685790'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:25.893901 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-86bc0fb5-3b2e-4bc1-bbbc-f1b1f430de69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '624e8e4513b449cfaf0343e614685790'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:25.894859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-86bc0fb5-3b2e-4bc1-bbbc-f1b1f430de69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '624e8e4513b449cfaf0343e614685790'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:25.986265 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 822/1649] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:33:25 2022] DELETE /identity/v3/users/624e8e4513b449cfaf0343e614685790 => generated 0 bytes in 770 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:25.989275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f173908-406c-4b38-b27f-1cd4fb5dbaf0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:25.989275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f173908-406c-4b38-b27f-1cd4fb5dbaf0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:25.989275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f173908-406c-4b38-b27f-1cd4fb5dbaf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:26.034531 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6f173908-406c-4b38-b27f-1cd4fb5dbaf0 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:26.039607 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6f173908-406c-4b38-b27f-1cd4fb5dbaf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:26.244206 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 828/1650] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:33:26.301866 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc0fb5-3b2e-4bc1-bbbc-f1b1f430de69 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:26.303218 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-86bc0fb5-3b2e-4bc1-bbbc-f1b1f430de69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:26.343147 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f173908-406c-4b38-b27f-1cd4fb5dbaf0 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:26.345765 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6f173908-406c-4b38-b27f-1cd4fb5dbaf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:26.376857 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f173908-406c-4b38-b27f-1cd4fb5dbaf0 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:26.383232 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3347ab52-cf1f-4ee4-8b7a-461c1ba12e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:26.387117 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347ab52-cf1f-4ee4-8b7a-461c1ba12e05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:26.388061 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347ab52-cf1f-4ee4-8b7a-461c1ba12e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:26.391873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347ab52-cf1f-4ee4-8b7a-461c1ba12e05 admin admin] PATH_INFO: `/v3/users/022543d7d45f4708b39587f8d79bff94` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:26.396478 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3347ab52-cf1f-4ee4-8b7a-461c1ba12e05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=022543d7d45f4708b39587f8d79bff94)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:26.403701 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3347ab52-cf1f-4ee4-8b7a-461c1ba12e05 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:26.458029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bc0fb5-3b2e-4bc1-bbbc-f1b1f430de69 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:26.470238 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9e10ac-5c79-4cf5-b97e-0b7cb62f55f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:26.471386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e10ac-5c79-4cf5-b97e-0b7cb62f55f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:26.471808 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e10ac-5c79-4cf5-b97e-0b7cb62f55f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:26.472376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9e10ac-5c79-4cf5-b97e-0b7cb62f55f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:26.490991 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:26.537938 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e10ac-5c79-4cf5-b97e-0b7cb62f55f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:26.540698 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9e10ac-5c79-4cf5-b97e-0b7cb62f55f6 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:26.584342 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 823/1651] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:26.711290 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3347ab52-cf1f-4ee4-8b7a-461c1ba12e05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '022543d7d45f4708b39587f8d79bff94'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:26.712029 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3347ab52-cf1f-4ee4-8b7a-461c1ba12e05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '022543d7d45f4708b39587f8d79bff94'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:26.740648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3347ab52-cf1f-4ee4-8b7a-461c1ba12e05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '022543d7d45f4708b39587f8d79bff94'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:26.741413 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3347ab52-cf1f-4ee4-8b7a-461c1ba12e05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '022543d7d45f4708b39587f8d79bff94'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:26.861030 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3347ab52-cf1f-4ee4-8b7a-461c1ba12e05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '022543d7d45f4708b39587f8d79bff94'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:26.873068 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:26.951151 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 829/1652] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:33:26 2022] DELETE /identity/v3/users/022543d7d45f4708b39587f8d79bff94 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:26.952938 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b91d06-3537-4492-af32-7712cba2b433 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:26.953237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b91d06-3537-4492-af32-7712cba2b433 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:26.954191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b91d06-3537-4492-af32-7712cba2b433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:26.985388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-14b91d06-3537-4492-af32-7712cba2b433 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:26.986657 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-14b91d06-3537-4492-af32-7712cba2b433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:27.086220 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 824/1653] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:27.103539 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3347ab52-cf1f-4ee4-8b7a-461c1ba12e05 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:27.111394 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3347ab52-cf1f-4ee4-8b7a-461c1ba12e05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:27.226892 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3347ab52-cf1f-4ee4-8b7a-461c1ba12e05 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:27.231988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c236d03-c26d-4039-99cc-5494fa39a7dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:27.234528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c236d03-c26d-4039-99cc-5494fa39a7dd service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:27.235116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c236d03-c26d-4039-99cc-5494fa39a7dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:27.235832 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c236d03-c26d-4039-99cc-5494fa39a7dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:27.286906 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c236d03-c26d-4039-99cc-5494fa39a7dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:27.296024 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c236d03-c26d-4039-99cc-5494fa39a7dd service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:27.316970 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 830/1654] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:33:28.013295 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b91d06-3537-4492-af32-7712cba2b433 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:28.014945 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-14b91d06-3537-4492-af32-7712cba2b433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:28.046918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b91d06-3537-4492-af32-7712cba2b433 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:28.063342 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd1175-518f-4e11-bc60-ed5c88c37485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:28.073860 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd1175-518f-4e11-bc60-ed5c88c37485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:28.074807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd1175-518f-4e11-bc60-ed5c88c37485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:28.075371 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd1175-518f-4e11-bc60-ed5c88c37485 admin admin] PATH_INFO: `/v3/projects/8ea8b7a5063c4f36b8df8fd5024259eb` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:28.081659 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd1175-518f-4e11-bc60-ed5c88c37485 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ea8b7a5063c4f36b8df8fd5024259eb)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:28.085548 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd1175-518f-4e11-bc60-ed5c88c37485 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:28.235590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-54cd1175-518f-4e11-bc60-ed5c88c37485 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ea8b7a5063c4f36b8df8fd5024259eb', 'request_id': 'req-54cd1175-518f-4e11-bc60-ed5c88c37485'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:28.244301 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-54cd1175-518f-4e11-bc60-ed5c88c37485 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ea8b7a5063c4f36b8df8fd5024259eb', 'request_id': 'req-54cd1175-518f-4e11-bc60-ed5c88c37485'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:28.263658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-54cd1175-518f-4e11-bc60-ed5c88c37485 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ea8b7a5063c4f36b8df8fd5024259eb', 'request_id': 'req-54cd1175-518f-4e11-bc60-ed5c88c37485'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:28.362932 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-54cd1175-518f-4e11-bc60-ed5c88c37485 admin admin] The token cache is being invalidate because project 8ea8b7a5063c4f36b8df8fd5024259eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:33:28.363238 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-54cd1175-518f-4e11-bc60-ed5c88c37485 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:28.365362 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-54cd1175-518f-4e11-bc60-ed5c88c37485 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:28.375230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 825/1655] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:33:28 2022] DELETE /identity/v3/projects/8ea8b7a5063c4f36b8df8fd5024259eb => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:28.376368 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c236d03-c26d-4039-99cc-5494fa39a7dd service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:28.386851 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c236d03-c26d-4039-99cc-5494fa39a7dd service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:28.563713 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81b73f-ab59-4b37-a011-5422d47a9be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:28.564773 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81b73f-ab59-4b37-a011-5422d47a9be9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:28.565210 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81b73f-ab59-4b37-a011-5422d47a9be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:28.565592 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81b73f-ab59-4b37-a011-5422d47a9be9 admin admin] PATH_INFO: `/v3/users/8812e4c2be15460ba28f4d4f17e1ee43` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:28.567446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b81b73f-ab59-4b37-a011-5422d47a9be9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8812e4c2be15460ba28f4d4f17e1ee43)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:28.576751 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b81b73f-ab59-4b37-a011-5422d47a9be9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:28.628364 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:28.831768 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3b81b73f-ab59-4b37-a011-5422d47a9be9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8812e4c2be15460ba28f4d4f17e1ee43'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:28.832484 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3b81b73f-ab59-4b37-a011-5422d47a9be9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8812e4c2be15460ba28f4d4f17e1ee43'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:28.849206 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3b81b73f-ab59-4b37-a011-5422d47a9be9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8812e4c2be15460ba28f4d4f17e1ee43'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:28.850053 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3b81b73f-ab59-4b37-a011-5422d47a9be9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8812e4c2be15460ba28f4d4f17e1ee43'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:28.919439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3b81b73f-ab59-4b37-a011-5422d47a9be9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8812e4c2be15460ba28f4d4f17e1ee43'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:28.931031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325d0ea-ec45-459d-8c11-369e4347a27a None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:28.931365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325d0ea-ec45-459d-8c11-369e4347a27a None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:28.931671 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e325d0ea-ec45-459d-8c11-369e4347a27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:28.959484 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 831/1656] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:33:28 2022] DELETE /identity/v3/users/8812e4c2be15460ba28f4d4f17e1ee43 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:28.960984 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81b73f-ab59-4b37-a011-5422d47a9be9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:28.962581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3b81b73f-ab59-4b37-a011-5422d47a9be9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:29.037128 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-e325d0ea-ec45-459d-8c11-369e4347a27a None None] MFA Rules not processed for user `fadae20ed280461e806b7b7d06c7b005`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:29.038389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-e325d0ea-ec45-459d-8c11-369e4347a27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:29.102124 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81b73f-ab59-4b37-a011-5422d47a9be9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:29.120907 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8bcc0-bf24-4a7b-adfd-ff6337466215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:29.122577 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8bcc0-bf24-4a7b-adfd-ff6337466215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:29.122941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8bcc0-bf24-4a7b-adfd-ff6337466215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:29.123333 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa8bcc0-bf24-4a7b-adfd-ff6337466215 admin admin] PATH_INFO: `/v3/projects/4c9afcac19364491989d502823267a41` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:29.125937 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8bcc0-bf24-4a7b-adfd-ff6337466215 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c9afcac19364491989d502823267a41)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:29.127083 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: WARNING py.warnings [None req-cfa8bcc0-bf24-4a7b-adfd-ff6337466215 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 30 10:33:29.127083 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: warnings.warn(msg) Jun 30 10:33:29.127083 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: Jun 30 10:33:29.132145 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa8bcc0-bf24-4a7b-adfd-ff6337466215 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:29.232623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:29.289197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 826/1657] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:28 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:29.376912 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cfa8bcc0-bf24-4a7b-adfd-ff6337466215 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c9afcac19364491989d502823267a41', 'request_id': 'req-cfa8bcc0-bf24-4a7b-adfd-ff6337466215'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:29.485258 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cfa8bcc0-bf24-4a7b-adfd-ff6337466215 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c9afcac19364491989d502823267a41', 'request_id': 'req-cfa8bcc0-bf24-4a7b-adfd-ff6337466215'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:29.493909 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cfa8bcc0-bf24-4a7b-adfd-ff6337466215 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c9afcac19364491989d502823267a41', 'request_id': 'req-cfa8bcc0-bf24-4a7b-adfd-ff6337466215'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:29.546012 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cfa8bcc0-bf24-4a7b-adfd-ff6337466215 admin admin] The token cache is being invalidate because project 4c9afcac19364491989d502823267a41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:33:29.546319 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cfa8bcc0-bf24-4a7b-adfd-ff6337466215 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:29.551899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-cfa8bcc0-bf24-4a7b-adfd-ff6337466215 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:29.557306 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 832/1658] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:33:28 2022] DELETE /identity/v3/projects/4c9afcac19364491989d502823267a41 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:29.567017 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325d0ea-ec45-459d-8c11-369e4347a27a None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:29.582954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e325d0ea-ec45-459d-8c11-369e4347a27a None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:29.755554 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f3735-fff6-47fe-bfbc-90501ecf709f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:29.756707 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f3735-fff6-47fe-bfbc-90501ecf709f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:29.757060 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f3735-fff6-47fe-bfbc-90501ecf709f admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:29.757468 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f3735-fff6-47fe-bfbc-90501ecf709f admin admin] PATH_INFO: `/v3/users/20757c9be9ad4b0aab725049b6a6f4e9` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:29.764839 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f3735-fff6-47fe-bfbc-90501ecf709f admin admin] RBAC: Authorizing `identity:delete_user(user_id=20757c9be9ad4b0aab725049b6a6f4e9)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:29.766939 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168f3735-fff6-47fe-bfbc-90501ecf709f admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:30.050369 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-168f3735-fff6-47fe-bfbc-90501ecf709f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20757c9be9ad4b0aab725049b6a6f4e9'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:30.199081 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-168f3735-fff6-47fe-bfbc-90501ecf709f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20757c9be9ad4b0aab725049b6a6f4e9'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:30.223978 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-168f3735-fff6-47fe-bfbc-90501ecf709f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20757c9be9ad4b0aab725049b6a6f4e9'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:30.225011 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-168f3735-fff6-47fe-bfbc-90501ecf709f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20757c9be9ad4b0aab725049b6a6f4e9'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:30.225997 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-168f3735-fff6-47fe-bfbc-90501ecf709f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20757c9be9ad4b0aab725049b6a6f4e9'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:30.271821 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 827/1659] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:33:29 2022] DELETE /identity/v3/users/20757c9be9ad4b0aab725049b6a6f4e9 => generated 0 bytes in 705 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:31.147680 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8bcc0-bf24-4a7b-adfd-ff6337466215 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:31.149359 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cfa8bcc0-bf24-4a7b-adfd-ff6337466215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:31.177934 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa8bcc0-bf24-4a7b-adfd-ff6337466215 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:31.238892 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:31.297816 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d395b2cd-fae5-4e08-859d-dd1a6422138c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:31.299235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395b2cd-fae5-4e08-859d-dd1a6422138c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:31.299753 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395b2cd-fae5-4e08-859d-dd1a6422138c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:31.300246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d395b2cd-fae5-4e08-859d-dd1a6422138c admin admin] PATH_INFO: `/v3/projects/3fa369cb54eb481b8b97593178e6daa5` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:31.301739 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d395b2cd-fae5-4e08-859d-dd1a6422138c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fa369cb54eb481b8b97593178e6daa5)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:31.303979 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d395b2cd-fae5-4e08-859d-dd1a6422138c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:31.376046 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f3735-fff6-47fe-bfbc-90501ecf709f admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:31.377964 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-168f3735-fff6-47fe-bfbc-90501ecf709f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:31.424592 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168f3735-fff6-47fe-bfbc-90501ecf709f admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:31.448323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d395b2cd-fae5-4e08-859d-dd1a6422138c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fa369cb54eb481b8b97593178e6daa5', 'request_id': 'req-d395b2cd-fae5-4e08-859d-dd1a6422138c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:31.527533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ff93cb-748e-4629-9f99-7e14766f854b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:31.529021 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff93cb-748e-4629-9f99-7e14766f854b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:31.529771 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff93cb-748e-4629-9f99-7e14766f854b admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:31.531660 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ff93cb-748e-4629-9f99-7e14766f854b admin admin] PATH_INFO: `/v3/users/30ffcd735910410fa38e6d24e19fd7e2` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:31.533700 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff93cb-748e-4629-9f99-7e14766f854b admin admin] RBAC: Authorizing `identity:delete_user(user_id=30ffcd735910410fa38e6d24e19fd7e2)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:31.536113 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ff93cb-748e-4629-9f99-7e14766f854b admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:31.540893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d395b2cd-fae5-4e08-859d-dd1a6422138c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fa369cb54eb481b8b97593178e6daa5', 'request_id': 'req-d395b2cd-fae5-4e08-859d-dd1a6422138c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:31.548511 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d395b2cd-fae5-4e08-859d-dd1a6422138c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fa369cb54eb481b8b97593178e6daa5', 'request_id': 'req-d395b2cd-fae5-4e08-859d-dd1a6422138c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:31.551283 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:31.577059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d395b2cd-fae5-4e08-859d-dd1a6422138c admin admin] The token cache is being invalidate because project 3fa369cb54eb481b8b97593178e6daa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:33:31.577631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d395b2cd-fae5-4e08-859d-dd1a6422138c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:31.579693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d395b2cd-fae5-4e08-859d-dd1a6422138c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:31.582512 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 833/1660] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:33:31 2022] DELETE /identity/v3/projects/3fa369cb54eb481b8b97593178e6daa5 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:31.588912 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c365483-2d82-49a2-9437-c676a6c252e5 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:31.594278 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c365483-2d82-49a2-9437-c676a6c252e5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:31.595094 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c365483-2d82-49a2-9437-c676a6c252e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:31.627201 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8c365483-2d82-49a2-9437-c676a6c252e5 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:31.630083 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8c365483-2d82-49a2-9437-c676a6c252e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:31.651530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 834/1661] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:31.655177 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c71ed7-0713-4318-b526-bb6704cfb8c0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:31.655829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c71ed7-0713-4318-b526-bb6704cfb8c0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:31.657342 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c71ed7-0713-4318-b526-bb6704cfb8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:31.698716 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-08c71ed7-0713-4318-b526-bb6704cfb8c0 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:31.700258 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-08c71ed7-0713-4318-b526-bb6704cfb8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:31.718137 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 835/1662] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:31.720081 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c71ed7-0713-4318-b526-bb6704cfb8c0 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:31.740894 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c71ed7-0713-4318-b526-bb6704cfb8c0 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:31.788139 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94d97f-54f2-43c0-8db0-99f28d94d614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:31.792949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94d97f-54f2-43c0-8db0-99f28d94d614 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:31.793240 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94d97f-54f2-43c0-8db0-99f28d94d614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:31.793525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f94d97f-54f2-43c0-8db0-99f28d94d614 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:31.810769 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f94d97f-54f2-43c0-8db0-99f28d94d614 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:33:31.811476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f94d97f-54f2-43c0-8db0-99f28d94d614 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:31.829703 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f94d97f-54f2-43c0-8db0-99f28d94d614 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:31.830546 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-17ff93cb-748e-4629-9f99-7e14766f854b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30ffcd735910410fa38e6d24e19fd7e2'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:31.863444 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 836/1663] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:33:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:33:31.869825 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94d97f-54f2-43c0-8db0-99f28d94d614 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:31.908035 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f94d97f-54f2-43c0-8db0-99f28d94d614 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:31.923985 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d49592-9bda-43f9-9fdc-e36abb447709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:31.924611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d49592-9bda-43f9-9fdc-e36abb447709 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:31.925397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d49592-9bda-43f9-9fdc-e36abb447709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:31.925397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d49592-9bda-43f9-9fdc-e36abb447709 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:31.925878 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d49592-9bda-43f9-9fdc-e36abb447709 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:31.926405 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-17ff93cb-748e-4629-9f99-7e14766f854b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30ffcd735910410fa38e6d24e19fd7e2'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:31.928073 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d49592-9bda-43f9-9fdc-e36abb447709 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:31.957470 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-17ff93cb-748e-4629-9f99-7e14766f854b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30ffcd735910410fa38e6d24e19fd7e2'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:31.958576 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-17ff93cb-748e-4629-9f99-7e14766f854b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30ffcd735910410fa38e6d24e19fd7e2'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:31.959451 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-17ff93cb-748e-4629-9f99-7e14766f854b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30ffcd735910410fa38e6d24e19fd7e2'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:31.996676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 828/1664] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:33:31 2022] DELETE /identity/v3/users/30ffcd735910410fa38e6d24e19fd7e2 => generated 0 bytes in 621 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:32.001674 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4dfe2-1a44-48f9-87b4-43f193b0719a None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:32.002400 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4dfe2-1a44-48f9-87b4-43f193b0719a None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:32.006205 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f4dfe2-1a44-48f9-87b4-43f193b0719a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:32.051251 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 837/1665] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:33:31 2022] POST /identity/v3/projects => generated 375 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:32.055131 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d49592-9bda-43f9-9fdc-e36abb447709 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:32.057615 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e2d49592-9bda-43f9-9fdc-e36abb447709 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:32.059021 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-72f4dfe2-1a44-48f9-87b4-43f193b0719a None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:32.060418 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-72f4dfe2-1a44-48f9-87b4-43f193b0719a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:32.224170 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 829/1666] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:32.227543 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d49592-9bda-43f9-9fdc-e36abb447709 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:32.228313 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c44b6d-3101-4e34-9a70-95e6c132bb46 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:32.228660 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c44b6d-3101-4e34-9a70-95e6c132bb46 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:32.229454 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c44b6d-3101-4e34-9a70-95e6c132bb46 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:32.239420 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 830/1667] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jun 30 10:33:32 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:33:32.239851 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43757195-4d42-4777-951d-85a40124a33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:32.240626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43757195-4d42-4777-951d-85a40124a33d admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:32.241098 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43757195-4d42-4777-951d-85a40124a33d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:32.241519 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43757195-4d42-4777-951d-85a40124a33d admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:32.242600 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43757195-4d42-4777-951d-85a40124a33d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:32.245015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43757195-4d42-4777-951d-85a40124a33d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:32.265471 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c44b6d-3101-4e34-9a70-95e6c132bb46 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:32.267573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-64c44b6d-3101-4e34-9a70-95e6c132bb46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:32.381401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 838/1668] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:33:32 2022] POST /identity/v3/users => generated 288 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:33:32.382841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebdb28-5956-4fa8-83e9-a16267264a9b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:32.383139 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebdb28-5956-4fa8-83e9-a16267264a9b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:32.383489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebdb28-5956-4fa8-83e9-a16267264a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:32.417166 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c44b6d-3101-4e34-9a70-95e6c132bb46 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:32.420806 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4284406-8174-4799-a11b-65390bff0ee4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:32.421489 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4284406-8174-4799-a11b-65390bff0ee4 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:32.421831 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4284406-8174-4799-a11b-65390bff0ee4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:32.422117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4284406-8174-4799-a11b-65390bff0ee4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:32.435167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ebebdb28-5956-4fa8-83e9-a16267264a9b None None] MFA Rules not processed for user `46c99f43a9344d7d92632990e45bb9cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:32.437925 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ebebdb28-5956-4fa8-83e9-a16267264a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:32.444592 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4284406-8174-4799-a11b-65390bff0ee4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:32.448564 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4284406-8174-4799-a11b-65390bff0ee4 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:32.468856 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 831/1669] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:33:32.470275 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4284406-8174-4799-a11b-65390bff0ee4 service neutron] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:32.482185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4284406-8174-4799-a11b-65390bff0ee4 service neutron] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:32.488619 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a9636-52a3-438b-a1d8-a10bf6414cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:32.495526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a9636-52a3-438b-a1d8-a10bf6414cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:32.495526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a9636-52a3-438b-a1d8-a10bf6414cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:32.495526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8a9636-52a3-438b-a1d8-a10bf6414cb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:32.495526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8a9636-52a3-438b-a1d8-a10bf6414cb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:32.495526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8a9636-52a3-438b-a1d8-a10bf6414cb7 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:32.510882 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 832/1670] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:32.516035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a9636-52a3-438b-a1d8-a10bf6414cb7 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:32.535241 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8a9636-52a3-438b-a1d8-a10bf6414cb7 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:32.542868 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7889e2e-7730-40c7-9453-5c98c8426f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:32.544073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7889e2e-7730-40c7-9453-5c98c8426f63 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:32.544990 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7889e2e-7730-40c7-9453-5c98c8426f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:32.545418 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7889e2e-7730-40c7-9453-5c98c8426f63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:32.546398 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7889e2e-7730-40c7-9453-5c98c8426f63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:32.548244 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7889e2e-7730-40c7-9453-5c98c8426f63 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:32.583407 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 833/1671] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:32 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:32.583407 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7889e2e-7730-40c7-9453-5c98c8426f63 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:32.593549 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7889e2e-7730-40c7-9453-5c98c8426f63 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:32.601818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 839/1672] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jun 30 10:33:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:32.604063 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ae1e41-08bc-4707-8000-291149386b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:32.605629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae1e41-08bc-4707-8000-291149386b98 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:32.606077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae1e41-08bc-4707-8000-291149386b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:32.606488 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae1e41-08bc-4707-8000-291149386b98 admin admin] PATH_INFO: `/v3/projects/3a5f238764b845f7bcb17cd15935b51b/users/b4da4c2e2cb743b89344c49b30deedc7/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:32.627786 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ae1e41-08bc-4707-8000-291149386b98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a5f238764b845f7bcb17cd15935b51b, user_id=b4da4c2e2cb743b89344c49b30deedc7, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:32.631071 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ae1e41-08bc-4707-8000-291149386b98 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:32.722295 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 834/1673] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:32 2022] PUT /identity/v3/projects/3a5f238764b845f7bcb17cd15935b51b/users/b4da4c2e2cb743b89344c49b30deedc7/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:32.733026 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e03c2-fb08-4089-9634-9e959b29ef9f None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:32.734971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c9c11-ee3f-4b4a-8537-82fcee97e62b None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:32.735351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c9c11-ee3f-4b4a-8537-82fcee97e62b None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:32.735777 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e03c2-fb08-4089-9634-9e959b29ef9f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:32.736122 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c9c11-ee3f-4b4a-8537-82fcee97e62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:32.737911 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525e03c2-fb08-4089-9634-9e959b29ef9f None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:32.740324 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 840/1674] 10.222.0.47 () {54 vars in 960 bytes} [Thu Jun 30 10:33:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:33:32.748640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99db6b-fe29-4534-a10e-9d61f444a0ab None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:32.748927 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99db6b-fe29-4534-a10e-9d61f444a0ab None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:32.749243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f99db6b-fe29-4534-a10e-9d61f444a0ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:32.810746 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8f99db6b-fe29-4534-a10e-9d61f444a0ab None None] MFA Rules not processed for user `46c99f43a9344d7d92632990e45bb9cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:32.811772 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8f99db6b-fe29-4534-a10e-9d61f444a0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:32.817237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-390c9c11-ee3f-4b4a-8537-82fcee97e62b None None] MFA Rules not processed for user `b4da4c2e2cb743b89344c49b30deedc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:32.818446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-390c9c11-ee3f-4b4a-8537-82fcee97e62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:33.050671 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 841/1675] 10.222.0.47 () {58 vars in 1057 bytes} [Thu Jun 30 10:33:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:33:33.052513 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99db6b-fe29-4534-a10e-9d61f444a0ab None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:33.054943 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8f99db6b-fe29-4534-a10e-9d61f444a0ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:33.069186 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 835/1676] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:33.069186 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c9c11-ee3f-4b4a-8537-82fcee97e62b None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:33.069539 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:33.110236 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c9c11-ee3f-4b4a-8537-82fcee97e62b None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:33.130647 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c98278-95f4-4434-bded-adb371de9794 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46c99f43a9344d7d92632990e45bb9cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:33.131366 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c98278-95f4-4434-bded-adb371de9794 service placement] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:33.133823 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c98278-95f4-4434-bded-adb371de9794 service placement] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:33.134129 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c98278-95f4-4434-bded-adb371de9794 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:33.156758 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c98278-95f4-4434-bded-adb371de9794 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:33.158635 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c98278-95f4-4434-bded-adb371de9794 service placement] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:33.186821 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 836/1677] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:33.186821 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c98278-95f4-4434-bded-adb371de9794 service placement] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:33.245762 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f99db6b-fe29-4534-a10e-9d61f444a0ab None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:33.253261 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f6fb8-5d8b-48f7-9ceb-7bdeb0beb4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:33.254330 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f6fb8-5d8b-48f7-9ceb-7bdeb0beb4ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:33.254613 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f6fb8-5d8b-48f7-9ceb-7bdeb0beb4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:33.254888 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f6fb8-5d8b-48f7-9ceb-7bdeb0beb4ff admin admin] PATH_INFO: `/v3/projects/bca1d43679df47768d3bbc39f584f497` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:33.256560 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f6fb8-5d8b-48f7-9ceb-7bdeb0beb4ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=bca1d43679df47768d3bbc39f584f497)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:33.261385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f6fb8-5d8b-48f7-9ceb-7bdeb0beb4ff admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:33.269780 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:33.349643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c98278-95f4-4434-bded-adb371de9794 service placement] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:33.369282 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009ff1d-2ac4-4ffa-acf1-599145f945b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:33.370161 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009ff1d-2ac4-4ffa-acf1-599145f945b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:33.370554 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009ff1d-2ac4-4ffa-acf1-599145f945b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:33.370975 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009ff1d-2ac4-4ffa-acf1-599145f945b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:33.371826 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009ff1d-2ac4-4ffa-acf1-599145f945b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:33.376985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009ff1d-2ac4-4ffa-acf1-599145f945b9 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:33.468110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-264f6fb8-5d8b-48f7-9ceb-7bdeb0beb4ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bca1d43679df47768d3bbc39f584f497', 'request_id': 'req-264f6fb8-5d8b-48f7-9ceb-7bdeb0beb4ff'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:33.547176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-264f6fb8-5d8b-48f7-9ceb-7bdeb0beb4ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bca1d43679df47768d3bbc39f584f497', 'request_id': 'req-264f6fb8-5d8b-48f7-9ceb-7bdeb0beb4ff'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:33.554044 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-264f6fb8-5d8b-48f7-9ceb-7bdeb0beb4ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bca1d43679df47768d3bbc39f584f497', 'request_id': 'req-264f6fb8-5d8b-48f7-9ceb-7bdeb0beb4ff'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:33.559218 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 837/1678] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:33:33 2022] POST /identity/v3/projects => generated 377 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:33.563216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009ff1d-2ac4-4ffa-acf1-599145f945b9 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:33.578283 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-264f6fb8-5d8b-48f7-9ceb-7bdeb0beb4ff admin admin] The token cache is being invalidate because project bca1d43679df47768d3bbc39f584f497 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:33:33.578771 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-264f6fb8-5d8b-48f7-9ceb-7bdeb0beb4ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:33.580471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-264f6fb8-5d8b-48f7-9ceb-7bdeb0beb4ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:33.584976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 842/1679] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:33:33 2022] DELETE /identity/v3/projects/bca1d43679df47768d3bbc39f584f497 => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:33.591560 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cedde-8b27-402e-868d-52dbad242231 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:33.593314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cedde-8b27-402e-868d-52dbad242231 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:33.593784 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cedde-8b27-402e-868d-52dbad242231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:33.638769 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-569cedde-8b27-402e-868d-52dbad242231 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:33.643096 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-569cedde-8b27-402e-868d-52dbad242231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:33.788096 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 843/1680] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:33.788199 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009ff1d-2ac4-4ffa-acf1-599145f945b9 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:33.790125 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260a29c-691f-4efb-ae6b-741602a30a6e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:33.790434 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260a29c-691f-4efb-ae6b-741602a30a6e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:33.790720 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b260a29c-691f-4efb-ae6b-741602a30a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:33.806515 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607a7f6-b57c-4478-9ef4-e5fdaddf25ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:33.809253 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607a7f6-b57c-4478-9ef4-e5fdaddf25ef admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:33.809547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607a7f6-b57c-4478-9ef4-e5fdaddf25ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:33.809843 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d607a7f6-b57c-4478-9ef4-e5fdaddf25ef admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:33.823304 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b260a29c-691f-4efb-ae6b-741602a30a6e None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:33.825755 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b260a29c-691f-4efb-ae6b-741602a30a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:33.829465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607a7f6-b57c-4478-9ef4-e5fdaddf25ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:33.842716 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d607a7f6-b57c-4478-9ef4-e5fdaddf25ef admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:33.847453 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 844/1681] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:33:33.848996 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b260a29c-691f-4efb-ae6b-741602a30a6e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:33.866824 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b260a29c-691f-4efb-ae6b-741602a30a6e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:33.885825 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c48905-8ebe-46b4-84a5-d91a521026b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:33.886465 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c48905-8ebe-46b4-84a5-d91a521026b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:33.887897 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c48905-8ebe-46b4-84a5-d91a521026b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:33.888735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c48905-8ebe-46b4-84a5-d91a521026b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:33.891058 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c48905-8ebe-46b4-84a5-d91a521026b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:33:33.891419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c48905-8ebe-46b4-84a5-d91a521026b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:33.896727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c48905-8ebe-46b4-84a5-d91a521026b9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:33.924890 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 845/1682] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:33:33.935128 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c48905-8ebe-46b4-84a5-d91a521026b9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:33.955702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c48905-8ebe-46b4-84a5-d91a521026b9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:33.970204 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ce3cd-d8f3-4ebd-8535-48887208d14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:33.974565 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ce3cd-d8f3-4ebd-8535-48887208d14e admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:33.974981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ce3cd-d8f3-4ebd-8535-48887208d14e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:33.975445 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ce3cd-d8f3-4ebd-8535-48887208d14e admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:33.976570 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ce3cd-d8f3-4ebd-8535-48887208d14e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:33.984333 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ce3cd-d8f3-4ebd-8535-48887208d14e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:33.989153 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 838/1683] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:33:33 2022] POST /identity/v3/users => generated 295 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:33:33.992815 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607a7f6-b57c-4478-9ef4-e5fdaddf25ef admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:33.997309 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d607a7f6-b57c-4478-9ef4-e5fdaddf25ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:34.032413 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d607a7f6-b57c-4478-9ef4-e5fdaddf25ef admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:34.047428 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139ef35-3cb2-4149-804f-e7dfd45b39dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:34.048236 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139ef35-3cb2-4149-804f-e7dfd45b39dd admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:34.052574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139ef35-3cb2-4149-804f-e7dfd45b39dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:34.053089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a139ef35-3cb2-4149-804f-e7dfd45b39dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:34.055354 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a139ef35-3cb2-4149-804f-e7dfd45b39dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:34.060753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a139ef35-3cb2-4149-804f-e7dfd45b39dd admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:34.093226 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 839/1684] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:33:34.097559 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139ef35-3cb2-4149-804f-e7dfd45b39dd admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:34.100546 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 846/1685] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:33:33 2022] POST /identity/v3/projects => generated 365 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:34.105705 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ce3cd-d8f3-4ebd-8535-48887208d14e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:34.111995 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a139ef35-3cb2-4149-804f-e7dfd45b39dd admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:34.231755 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc3b69-e513-43df-a740-e1d06e3102d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:34.234669 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc3b69-e513-43df-a740-e1d06e3102d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:34.235176 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc3b69-e513-43df-a740-e1d06e3102d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:34.235728 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc3b69-e513-43df-a740-e1d06e3102d5 admin admin] PATH_INFO: `/v3/projects/8421fba1161945f0964e23740fe9fa39/users/28cc5bf620d94228bcb5359e7d8f9aab/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:34.236568 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ce3cd-d8f3-4ebd-8535-48887208d14e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:34.247392 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a367c-371b-40a6-a22b-708d6253212c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:34.248513 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a367c-371b-40a6-a22b-708d6253212c admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:34.248513 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a367c-371b-40a6-a22b-708d6253212c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:34.248820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324a367c-371b-40a6-a22b-708d6253212c admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:34.250606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a367c-371b-40a6-a22b-708d6253212c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:34.257785 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324a367c-371b-40a6-a22b-708d6253212c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:34.264625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc3b69-e513-43df-a740-e1d06e3102d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8421fba1161945f0964e23740fe9fa39, user_id=28cc5bf620d94228bcb5359e7d8f9aab, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:34.270940 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc3b69-e513-43df-a740-e1d06e3102d5 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:34.354925 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 840/1686] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:34 2022] PUT /identity/v3/projects/8421fba1161945f0964e23740fe9fa39/users/28cc5bf620d94228bcb5359e7d8f9aab/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:33:34.359778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc3b69-e513-43df-a740-e1d06e3102d5 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:34.386402 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 847/1687] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:33:34 2022] POST /identity/v3/users => generated 283 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:33:34.389807 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a367c-371b-40a6-a22b-708d6253212c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:34.417974 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324a367c-371b-40a6-a22b-708d6253212c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:34.472486 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc3b69-e513-43df-a740-e1d06e3102d5 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:34.481420 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0338a8-5c5d-4c1f-967d-24f60cb4dc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:34.483297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0338a8-5c5d-4c1f-967d-24f60cb4dc57 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:34.483935 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0338a8-5c5d-4c1f-967d-24f60cb4dc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:34.484363 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0338a8-5c5d-4c1f-967d-24f60cb4dc57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:34.485251 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0338a8-5c5d-4c1f-967d-24f60cb4dc57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:34.492463 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0338a8-5c5d-4c1f-967d-24f60cb4dc57 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:34.516751 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 841/1688] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:34.520229 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0338a8-5c5d-4c1f-967d-24f60cb4dc57 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:34.532011 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0338a8-5c5d-4c1f-967d-24f60cb4dc57 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:34.537347 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:34.540885 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05944a5-fea1-44dc-8626-85417914c167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:34.544266 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05944a5-fea1-44dc-8626-85417914c167 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:34.544266 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05944a5-fea1-44dc-8626-85417914c167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:34.544266 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05944a5-fea1-44dc-8626-85417914c167 admin admin] PATH_INFO: `/v3/projects/8421fba1161945f0964e23740fe9fa39/users/28cc5bf620d94228bcb5359e7d8f9aab/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:34.546681 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655878e-545e-464d-a628-748e2344fc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:34.547416 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655878e-545e-464d-a628-748e2344fc6e admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:34.547710 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655878e-545e-464d-a628-748e2344fc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:34.547990 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6655878e-545e-464d-a628-748e2344fc6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:34.549059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6655878e-545e-464d-a628-748e2344fc6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:34.550777 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6655878e-545e-464d-a628-748e2344fc6e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:34.555547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05944a5-fea1-44dc-8626-85417914c167 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8421fba1161945f0964e23740fe9fa39, user_id=28cc5bf620d94228bcb5359e7d8f9aab, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:34.559211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05944a5-fea1-44dc-8626-85417914c167 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:34.564740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:34.570020 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 848/1689] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:34.577318 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655878e-545e-464d-a628-748e2344fc6e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:34.586083 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 842/1690] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:34 2022] PUT /identity/v3/projects/8421fba1161945f0964e23740fe9fa39/users/28cc5bf620d94228bcb5359e7d8f9aab/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:33:34.589257 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6655878e-545e-464d-a628-748e2344fc6e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:34.599512 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8ed06-ee16-4391-a1a9-b7e5c43442fa None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:34.600021 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8ed06-ee16-4391-a1a9-b7e5c43442fa None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:34.600445 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c8ed06-ee16-4391-a1a9-b7e5c43442fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:34.660398 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-07c8ed06-ee16-4391-a1a9-b7e5c43442fa None None] MFA Rules not processed for user `28cc5bf620d94228bcb5359e7d8f9aab`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:34.663504 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-07c8ed06-ee16-4391-a1a9-b7e5c43442fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:34.732887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfe24c-b94b-45f5-880f-de4023ddee08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:34.733594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfe24c-b94b-45f5-880f-de4023ddee08 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:34.733884 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfe24c-b94b-45f5-880f-de4023ddee08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:34.734169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcfe24c-b94b-45f5-880f-de4023ddee08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:34.734925 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcfe24c-b94b-45f5-880f-de4023ddee08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:34.736610 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcfe24c-b94b-45f5-880f-de4023ddee08 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:34.758077 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 849/1691] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:34 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:34.762645 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfe24c-b94b-45f5-880f-de4023ddee08 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:34.787963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcfe24c-b94b-45f5-880f-de4023ddee08 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:34.800270 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3597c73-6d84-443d-9c38-d92d783255b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:34.803807 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3597c73-6d84-443d-9c38-d92d783255b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:34.806857 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3597c73-6d84-443d-9c38-d92d783255b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:34.806857 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3597c73-6d84-443d-9c38-d92d783255b6 admin admin] PATH_INFO: `/v3/projects/87089848a6ea4d579579f1d3ebbd8965/users/63d912ca14c845dc9016d6b7da2b4b5d/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:34.850663 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 843/1692] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:34 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:34.852072 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3597c73-6d84-443d-9c38-d92d783255b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87089848a6ea4d579579f1d3ebbd8965, user_id=63d912ca14c845dc9016d6b7da2b4b5d, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:34.857488 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3597c73-6d84-443d-9c38-d92d783255b6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:34.859171 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8ed06-ee16-4391-a1a9-b7e5c43442fa None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:34.860740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-07c8ed06-ee16-4391-a1a9-b7e5c43442fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:34.949715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 850/1693] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:34 2022] PUT /identity/v3/projects/87089848a6ea4d579579f1d3ebbd8965/users/63d912ca14c845dc9016d6b7da2b4b5d/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:34.959130 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d2280-cd03-46d3-af99-b90278519f35 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:34.959130 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d2280-cd03-46d3-af99-b90278519f35 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:34.959596 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33d2280-cd03-46d3-af99-b90278519f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:35.023584 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f33d2280-cd03-46d3-af99-b90278519f35 None None] MFA Rules not processed for user `63d912ca14c845dc9016d6b7da2b4b5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:35.025563 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f33d2280-cd03-46d3-af99-b90278519f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:35.065428 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c8ed06-ee16-4391-a1a9-b7e5c43442fa None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:35.148508 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc904f-d0b0-4dd6-9323-7e7dc7230a9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:35.149106 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc904f-d0b0-4dd6-9323-7e7dc7230a9f service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:35.150997 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc904f-d0b0-4dd6-9323-7e7dc7230a9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:35.151729 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dc904f-d0b0-4dd6-9323-7e7dc7230a9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:35.156164 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-97dc904f-d0b0-4dd6-9323-7e7dc7230a9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:35.190264 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 851/1694] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:34 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:33:35.197820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d2280-cd03-46d3-af99-b90278519f35 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:35.245569 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dc904f-d0b0-4dd6-9323-7e7dc7230a9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:35.248239 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dc904f-d0b0-4dd6-9323-7e7dc7230a9f service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:35.263380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 844/1695] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:35.264623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc904f-d0b0-4dd6-9323-7e7dc7230a9f service cinder] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:35.266357 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-97dc904f-d0b0-4dd6-9323-7e7dc7230a9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:35.356455 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dc904f-d0b0-4dd6-9323-7e7dc7230a9f service cinder] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:35.373863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f83f8-2560-41c0-b9f8-5630bc40b438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:35.374863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f83f8-2560-41c0-b9f8-5630bc40b438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:35.375409 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f83f8-2560-41c0-b9f8-5630bc40b438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:35.375542 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f83f8-2560-41c0-b9f8-5630bc40b438 admin admin] PATH_INFO: `/v3/projects/f492cb5130db4570bd2a4445fa3128e6` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:35.376744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f83f8-2560-41c0-b9f8-5630bc40b438 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f492cb5130db4570bd2a4445fa3128e6)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:35.379508 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5f83f8-2560-41c0-b9f8-5630bc40b438 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:35.395545 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33d2280-cd03-46d3-af99-b90278519f35 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:35.423707 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4aaea0-e297-4a58-9462-326061f119ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:35.424408 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4aaea0-e297-4a58-9462-326061f119ce admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:35.425295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4aaea0-e297-4a58-9462-326061f119ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:35.425596 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4aaea0-e297-4a58-9462-326061f119ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:35.426325 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4aaea0-e297-4a58-9462-326061f119ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:35.428375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4aaea0-e297-4a58-9462-326061f119ce admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:35.518360 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:35.539692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1b5f83f8-2560-41c0-b9f8-5630bc40b438 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f492cb5130db4570bd2a4445fa3128e6', 'request_id': 'req-1b5f83f8-2560-41c0-b9f8-5630bc40b438'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:35.557693 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1b5f83f8-2560-41c0-b9f8-5630bc40b438 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f492cb5130db4570bd2a4445fa3128e6', 'request_id': 'req-1b5f83f8-2560-41c0-b9f8-5630bc40b438'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:35.577648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1b5f83f8-2560-41c0-b9f8-5630bc40b438 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f492cb5130db4570bd2a4445fa3128e6', 'request_id': 'req-1b5f83f8-2560-41c0-b9f8-5630bc40b438'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:35.598444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:35.603952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 852/1696] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:33:35 2022] POST /identity/v3/projects => generated 365 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:33:35.607055 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4aaea0-e297-4a58-9462-326061f119ce admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:35.679523 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1b5f83f8-2560-41c0-b9f8-5630bc40b438 admin admin] The token cache is being invalidate because project f492cb5130db4570bd2a4445fa3128e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:33:35.680528 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1b5f83f8-2560-41c0-b9f8-5630bc40b438 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:35.681138 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1b5f83f8-2560-41c0-b9f8-5630bc40b438 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:35.687306 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 845/1697] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:33:35 2022] DELETE /identity/v3/projects/f492cb5130db4570bd2a4445fa3128e6 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:35.691252 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f83f8-2560-41c0-b9f8-5630bc40b438 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:35.694478 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-1b5f83f8-2560-41c0-b9f8-5630bc40b438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:35.766648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4aaea0-e297-4a58-9462-326061f119ce admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:35.784999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c2b4f-2a01-43ef-891e-b9f37d6d1cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:35.785963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c2b4f-2a01-43ef-891e-b9f37d6d1cab admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:35.786411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c2b4f-2a01-43ef-891e-b9f37d6d1cab admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:35.786880 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632c2b4f-2a01-43ef-891e-b9f37d6d1cab admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:35.787929 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c2b4f-2a01-43ef-891e-b9f37d6d1cab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:35.791735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632c2b4f-2a01-43ef-891e-b9f37d6d1cab admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:35.872351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f83f8-2560-41c0-b9f8-5630bc40b438 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:35.886982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc10ee-b673-4cc7-92bb-8cb1f7606874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:35.888068 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc10ee-b673-4cc7-92bb-8cb1f7606874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:35.888248 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc10ee-b673-4cc7-92bb-8cb1f7606874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:35.889280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cc10ee-b673-4cc7-92bb-8cb1f7606874 admin admin] PATH_INFO: `/v3/users/27acf141898f45afb4b692dcba09b56f` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:35.889891 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc10ee-b673-4cc7-92bb-8cb1f7606874 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27acf141898f45afb4b692dcba09b56f)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:35.893055 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cc10ee-b673-4cc7-92bb-8cb1f7606874 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:35.959835 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 853/1698] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:33:35 2022] POST /identity/v3/users => generated 289 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:35.959835 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c2b4f-2a01-43ef-891e-b9f37d6d1cab admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:35.964127 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-632c2b4f-2a01-43ef-891e-b9f37d6d1cab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:35.997452 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632c2b4f-2a01-43ef-891e-b9f37d6d1cab admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:36.015908 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db55a89e-3a27-4dc9-8999-305a37ed10b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:36.016841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55a89e-3a27-4dc9-8999-305a37ed10b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:36.017774 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55a89e-3a27-4dc9-8999-305a37ed10b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:36.018072 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db55a89e-3a27-4dc9-8999-305a37ed10b3 admin admin] PATH_INFO: `/v3/users/4cc6a79de538411ea88772cf5cc3bc1f` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:36.030445 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db55a89e-3a27-4dc9-8999-305a37ed10b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cc6a79de538411ea88772cf5cc3bc1f)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:36.032503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db55a89e-3a27-4dc9-8999-305a37ed10b3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:36.134624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a9cc10ee-b673-4cc7-92bb-8cb1f7606874 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27acf141898f45afb4b692dcba09b56f'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:36.255415 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a9cc10ee-b673-4cc7-92bb-8cb1f7606874 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27acf141898f45afb4b692dcba09b56f'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:36.284410 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a9cc10ee-b673-4cc7-92bb-8cb1f7606874 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27acf141898f45afb4b692dcba09b56f'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:36.285184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a9cc10ee-b673-4cc7-92bb-8cb1f7606874 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27acf141898f45afb4b692dcba09b56f'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:36.285813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a9cc10ee-b673-4cc7-92bb-8cb1f7606874 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27acf141898f45afb4b692dcba09b56f'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:36.315199 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 846/1699] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:33:35 2022] DELETE /identity/v3/users/27acf141898f45afb4b692dcba09b56f => generated 0 bytes in 627 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:36.316523 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc10ee-b673-4cc7-92bb-8cb1f7606874 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:36.321313 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-a9cc10ee-b673-4cc7-92bb-8cb1f7606874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:36.324921 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-db55a89e-3a27-4dc9-8999-305a37ed10b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cc6a79de538411ea88772cf5cc3bc1f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:36.326952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-db55a89e-3a27-4dc9-8999-305a37ed10b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cc6a79de538411ea88772cf5cc3bc1f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:36.351958 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-db55a89e-3a27-4dc9-8999-305a37ed10b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cc6a79de538411ea88772cf5cc3bc1f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:36.355921 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-db55a89e-3a27-4dc9-8999-305a37ed10b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cc6a79de538411ea88772cf5cc3bc1f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:36.437211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-db55a89e-3a27-4dc9-8999-305a37ed10b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cc6a79de538411ea88772cf5cc3bc1f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:36.517236 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 854/1700] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:33:35 2022] DELETE /identity/v3/users/4cc6a79de538411ea88772cf5cc3bc1f => generated 0 bytes in 556 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:36.519385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd69197-e090-497a-8ef0-4f483a374906 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:36.519898 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd69197-e090-497a-8ef0-4f483a374906 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:36.520562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd69197-e090-497a-8ef0-4f483a374906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:36.548149 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cc10ee-b673-4cc7-92bb-8cb1f7606874 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:36.655505 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-afd69197-e090-497a-8ef0-4f483a374906 None None] MFA Rules not processed for user `c7f8abef711c4cd2b53852b3a969b120`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:36.657410 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-afd69197-e090-497a-8ef0-4f483a374906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:36.705637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:36.777484 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:36.801881 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7aa97-e338-41b7-a852-0e1d3d75c950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:36.802585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7aa97-e338-41b7-a852-0e1d3d75c950 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:36.802874 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7aa97-e338-41b7-a852-0e1d3d75c950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:36.803156 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7aa97-e338-41b7-a852-0e1d3d75c950 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:36.803940 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b7aa97-e338-41b7-a852-0e1d3d75c950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:36.805589 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b7aa97-e338-41b7-a852-0e1d3d75c950 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:36.871215 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 847/1701] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 553 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:36.871215 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7aa97-e338-41b7-a852-0e1d3d75c950 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:36.877017 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d7b7aa97-e338-41b7-a852-0e1d3d75c950 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:36.929408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7aa97-e338-41b7-a852-0e1d3d75c950 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:36.929818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 855/1702] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:36 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:36.947052 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca70a6-af9c-4cb6-9dde-96f352cf932f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:36.949091 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca70a6-af9c-4cb6-9dde-96f352cf932f admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:36.949388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca70a6-af9c-4cb6-9dde-96f352cf932f admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:36.949721 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca70a6-af9c-4cb6-9dde-96f352cf932f admin admin] PATH_INFO: `/v3/projects/13a33103e5fb468185777067be2d2240/users/f0e1c6cb69064c409f72b10f9dfc3a79/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:37.001650 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca70a6-af9c-4cb6-9dde-96f352cf932f admin admin] RBAC: Authorizing `identity:create_grant(project_id=13a33103e5fb468185777067be2d2240, user_id=f0e1c6cb69064c409f72b10f9dfc3a79, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:37.008269 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca70a6-af9c-4cb6-9dde-96f352cf932f admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:37.034978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ad401-f956-4c49-9b82-4b7d150969af None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:37.035354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ad401-f956-4c49-9b82-4b7d150969af None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:37.035562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03ad401-f956-4c49-9b82-4b7d150969af None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:37.037173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 856/1703] 10.222.0.47 () {54 vars in 961 bytes} [Thu Jun 30 10:33:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:33:37.038580 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:37.066996 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cfdbc2-ff15-4d98-a282-b586859e1f40 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:37.071751 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cfdbc2-ff15-4d98-a282-b586859e1f40 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:37.072811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cfdbc2-ff15-4d98-a282-b586859e1f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:37.108670 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 848/1704] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:36 2022] PUT /identity/v3/projects/13a33103e5fb468185777067be2d2240/users/f0e1c6cb69064c409f72b10f9dfc3a79/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:37.109925 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a8cfdbc2-ff15-4d98-a282-b586859e1f40 None None] MFA Rules not processed for user `71f6d7715e6f49ffb9fdf6e1307bbd51`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:37.111659 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a8cfdbc2-ff15-4d98-a282-b586859e1f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:37.112733 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca70a6-af9c-4cb6-9dde-96f352cf932f admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:37.218945 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca70a6-af9c-4cb6-9dde-96f352cf932f admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:37.224865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4b17d-aa19-4a4b-9f6a-29dcfeda85ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:37.225945 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4b17d-aa19-4a4b-9f6a-29dcfeda85ac admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:37.230887 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4b17d-aa19-4a4b-9f6a-29dcfeda85ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:37.230887 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa4b17d-aa19-4a4b-9f6a-29dcfeda85ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:37.230887 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4b17d-aa19-4a4b-9f6a-29dcfeda85ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:37.230887 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa4b17d-aa19-4a4b-9f6a-29dcfeda85ac admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:37.243298 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 849/1705] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:37 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:37.248917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4b17d-aa19-4a4b-9f6a-29dcfeda85ac admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:37.263303 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa4b17d-aa19-4a4b-9f6a-29dcfeda85ac admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:37.281374 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c6c94-7f2a-43df-b3c3-921b81433a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:37.282879 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c6c94-7f2a-43df-b3c3-921b81433a0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:37.283212 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c6c94-7f2a-43df-b3c3-921b81433a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:37.283524 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c6c94-7f2a-43df-b3c3-921b81433a0e admin admin] PATH_INFO: `/v3/projects/13a33103e5fb468185777067be2d2240/users/f0e1c6cb69064c409f72b10f9dfc3a79/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:37.287669 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39c6c94-7f2a-43df-b3c3-921b81433a0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=13a33103e5fb468185777067be2d2240, user_id=f0e1c6cb69064c409f72b10f9dfc3a79, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:37.296184 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 857/1706] 10.222.0.47 () {58 vars in 1058 bytes} [Thu Jun 30 10:33:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:33:37.304315 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39c6c94-7f2a-43df-b3c3-921b81433a0e admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:37.309821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cfdbc2-ff15-4d98-a282-b586859e1f40 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:37.340018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cfdbc2-ff15-4d98-a282-b586859e1f40 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:37.366491 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 850/1707] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:37 2022] PUT /identity/v3/projects/13a33103e5fb468185777067be2d2240/users/f0e1c6cb69064c409f72b10f9dfc3a79/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:37.369829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc2f05-9c04-4a51-94fb-8f09f8acf7ee None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:37.370264 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc2f05-9c04-4a51-94fb-8f09f8acf7ee None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:37.370664 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdc2f05-9c04-4a51-94fb-8f09f8acf7ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:37.454095 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-9bdc2f05-9c04-4a51-94fb-8f09f8acf7ee None None] MFA Rules not processed for user `797a7b1e128f4102bac6430a60d2c0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:37.460093 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-9bdc2f05-9c04-4a51-94fb-8f09f8acf7ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:37.466850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c096679-76c2-4528-9a2d-5b58f3fe1130 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:37.467748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c096679-76c2-4528-9a2d-5b58f3fe1130 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:37.470007 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c096679-76c2-4528-9a2d-5b58f3fe1130 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:37.470007 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c096679-76c2-4528-9a2d-5b58f3fe1130 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:37.556337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 851/1708] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:37.560922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057590bb-402e-4979-bfff-9475e13b8e92 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:37.562788 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057590bb-402e-4979-bfff-9475e13b8e92 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:37.563228 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057590bb-402e-4979-bfff-9475e13b8e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:37.608730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c096679-76c2-4528-9a2d-5b58f3fe1130 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:37.615406 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c096679-76c2-4528-9a2d-5b58f3fe1130 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:37.655184 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 858/1709] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:37.657849 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c96bdb-c6ae-4ff1-aa34-a6a9e09f782a None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:37.661301 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c96bdb-c6ae-4ff1-aa34-a6a9e09f782a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:37.661651 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c96bdb-c6ae-4ff1-aa34-a6a9e09f782a None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:37.669773 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-057590bb-402e-4979-bfff-9475e13b8e92 None None] MFA Rules not processed for user `f0e1c6cb69064c409f72b10f9dfc3a79`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:37.672221 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 859/1710] 10.222.0.47 () {54 vars in 961 bytes} [Thu Jun 30 10:33:37 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:33:37.673730 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-057590bb-402e-4979-bfff-9475e13b8e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:37.676581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174e2d0-fc01-4b1d-9f3a-e61143cd709f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:37.677187 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174e2d0-fc01-4b1d-9f3a-e61143cd709f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:37.677478 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174e2d0-fc01-4b1d-9f3a-e61143cd709f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:37.773894 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6174e2d0-fc01-4b1d-9f3a-e61143cd709f None None] MFA Rules not processed for user `71f6d7715e6f49ffb9fdf6e1307bbd51`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:37.777868 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6174e2d0-fc01-4b1d-9f3a-e61143cd709f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:37.829360 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 860/1711] 10.222.0.47 () {58 vars in 1058 bytes} [Thu Jun 30 10:33:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:37.839468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174e2d0-fc01-4b1d-9f3a-e61143cd709f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:37.861976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:37.879549 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174e2d0-fc01-4b1d-9f3a-e61143cd709f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:37.908177 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150f94c-c2da-4980-91b6-9ed30bf8363b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:37.909891 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150f94c-c2da-4980-91b6-9ed30bf8363b service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:37.911742 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150f94c-c2da-4980-91b6-9ed30bf8363b service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:37.912072 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150f94c-c2da-4980-91b6-9ed30bf8363b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:37.946267 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150f94c-c2da-4980-91b6-9ed30bf8363b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:37.950620 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150f94c-c2da-4980-91b6-9ed30bf8363b service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:37.996826 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 852/1712] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:37 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 435 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:38.012388 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 861/1713] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:37 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:38.014363 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057590bb-402e-4979-bfff-9475e13b8e92 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:38.018391 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-057590bb-402e-4979-bfff-9475e13b8e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:38.192412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057590bb-402e-4979-bfff-9475e13b8e92 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:38.211536 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150f94c-c2da-4980-91b6-9ed30bf8363b service nova] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:38.211536 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0150f94c-c2da-4980-91b6-9ed30bf8363b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:38.221695 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d4bf4-ed60-4886-b483-5aa85b71b031 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:38.224712 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d4bf4-ed60-4886-b483-5aa85b71b031 service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:38.226841 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d4bf4-ed60-4886-b483-5aa85b71b031 service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:38.227330 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d4bf4-ed60-4886-b483-5aa85b71b031 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:38.243838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d4bf4-ed60-4886-b483-5aa85b71b031 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:38.246993 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d4bf4-ed60-4886-b483-5aa85b71b031 service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:38.264279 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 853/1714] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:38 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:38.405521 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150f94c-c2da-4980-91b6-9ed30bf8363b service nova] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:38.412647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789a24d9-bad8-4ad4-88f1-16f4de89abdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:38.413579 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a24d9-bad8-4ad4-88f1-16f4de89abdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:38.413854 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a24d9-bad8-4ad4-88f1-16f4de89abdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:38.414136 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789a24d9-bad8-4ad4-88f1-16f4de89abdf admin admin] PATH_INFO: `/v3/projects/7547a6aa33724b48a45578255410924c` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:38.415478 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a24d9-bad8-4ad4-88f1-16f4de89abdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=7547a6aa33724b48a45578255410924c)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:38.418167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789a24d9-bad8-4ad4-88f1-16f4de89abdf admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:38.533821 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d4bf4-ed60-4886-b483-5aa85b71b031 service cinder] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:38.541121 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-278d4bf4-ed60-4886-b483-5aa85b71b031 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:38.582630 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d4bf4-ed60-4886-b483-5aa85b71b031 service cinder] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:38.620410 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-789a24d9-bad8-4ad4-88f1-16f4de89abdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7547a6aa33724b48a45578255410924c', 'request_id': 'req-789a24d9-bad8-4ad4-88f1-16f4de89abdf'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:38.720876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-789a24d9-bad8-4ad4-88f1-16f4de89abdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7547a6aa33724b48a45578255410924c', 'request_id': 'req-789a24d9-bad8-4ad4-88f1-16f4de89abdf'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:38.740270 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-789a24d9-bad8-4ad4-88f1-16f4de89abdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7547a6aa33724b48a45578255410924c', 'request_id': 'req-789a24d9-bad8-4ad4-88f1-16f4de89abdf'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:38.767245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55539b-5cbf-4670-9192-5f92f9822974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:38.773512 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-789a24d9-bad8-4ad4-88f1-16f4de89abdf admin admin] The token cache is being invalidate because project 7547a6aa33724b48a45578255410924c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:33:38.773512 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-789a24d9-bad8-4ad4-88f1-16f4de89abdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:38.779650 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-789a24d9-bad8-4ad4-88f1-16f4de89abdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:38.780427 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55539b-5cbf-4670-9192-5f92f9822974 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:38.780731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55539b-5cbf-4670-9192-5f92f9822974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:38.781273 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad55539b-5cbf-4670-9192-5f92f9822974 admin admin] PATH_INFO: `/v3/projects/c6c9c2c365864c2592431906f83c8e94` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:38.786422 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55539b-5cbf-4670-9192-5f92f9822974 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6c9c2c365864c2592431906f83c8e94)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:38.786875 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 862/1715] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:33:38 2022] DELETE /identity/v3/projects/7547a6aa33724b48a45578255410924c => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:33:38.786976 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad55539b-5cbf-4670-9192-5f92f9822974 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:38.788615 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff467c8-5c11-4270-b3ec-941a3503d6ab None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:38.791534 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff467c8-5c11-4270-b3ec-941a3503d6ab None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:38.806435 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff467c8-5c11-4270-b3ec-941a3503d6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:38.872195 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6ff467c8-5c11-4270-b3ec-941a3503d6ab None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:38.875331 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6ff467c8-5c11-4270-b3ec-941a3503d6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:38.886531 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 863/1716] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:38.898780 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:38.909953 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586982a-d5f7-4d6d-a5ad-b50cdecc547a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:38.910307 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586982a-d5f7-4d6d-a5ad-b50cdecc547a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:38.910625 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f586982a-d5f7-4d6d-a5ad-b50cdecc547a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:38.933402 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f586982a-d5f7-4d6d-a5ad-b50cdecc547a None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:38.935906 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f586982a-d5f7-4d6d-a5ad-b50cdecc547a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:38.951908 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 864/1717] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:38.956578 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586982a-d5f7-4d6d-a5ad-b50cdecc547a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:38.969380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f586982a-d5f7-4d6d-a5ad-b50cdecc547a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:38.975588 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2848a-f6fb-42e1-9d8b-780f030014b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:38.976605 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2848a-f6fb-42e1-9d8b-780f030014b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:38.977056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2848a-f6fb-42e1-9d8b-780f030014b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:38.977488 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a2848a-f6fb-42e1-9d8b-780f030014b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:38.979049 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a2848a-f6fb-42e1-9d8b-780f030014b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:33:38.979908 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a2848a-f6fb-42e1-9d8b-780f030014b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:38.998309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a2848a-f6fb-42e1-9d8b-780f030014b2 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:39.031640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 865/1718] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:33:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:33:39.036217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2848a-f6fb-42e1-9d8b-780f030014b2 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:39.045923 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-ad55539b-5cbf-4670-9192-5f92f9822974 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6c9c2c365864c2592431906f83c8e94', 'request_id': 'req-ad55539b-5cbf-4670-9192-5f92f9822974'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:39.060796 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-ad55539b-5cbf-4670-9192-5f92f9822974 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6c9c2c365864c2592431906f83c8e94', 'request_id': 'req-ad55539b-5cbf-4670-9192-5f92f9822974'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:39.073094 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a2848a-f6fb-42e1-9d8b-780f030014b2 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:39.081376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-ad55539b-5cbf-4670-9192-5f92f9822974 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6c9c2c365864c2592431906f83c8e94', 'request_id': 'req-ad55539b-5cbf-4670-9192-5f92f9822974'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:39.213013 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94fbe5d-0490-4cb5-9089-708ab13f461e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:39.214411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94fbe5d-0490-4cb5-9089-708ab13f461e admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:39.218786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94fbe5d-0490-4cb5-9089-708ab13f461e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:39.218786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94fbe5d-0490-4cb5-9089-708ab13f461e admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:39.221742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-ad55539b-5cbf-4670-9192-5f92f9822974 admin admin] The token cache is being invalidate because project c6c9c2c365864c2592431906f83c8e94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:33:39.221870 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-ad55539b-5cbf-4670-9192-5f92f9822974 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:39.222482 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94fbe5d-0490-4cb5-9089-708ab13f461e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:39.223218 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-ad55539b-5cbf-4670-9192-5f92f9822974 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:39.230275 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 854/1719] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:33:38 2022] DELETE /identity/v3/projects/c6c9c2c365864c2592431906f83c8e94 => generated 0 bytes in 694 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:39.230424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94fbe5d-0490-4cb5-9089-708ab13f461e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:39.239324 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55539b-5cbf-4670-9192-5f92f9822974 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:39.246188 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-ad55539b-5cbf-4670-9192-5f92f9822974 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:39.286061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad55539b-5cbf-4670-9192-5f92f9822974 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:39.295061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f6a85-90f8-44bd-98d8-f45f2638b2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:39.296037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f6a85-90f8-44bd-98d8-f45f2638b2e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:39.297062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f6a85-90f8-44bd-98d8-f45f2638b2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:39.297062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072f6a85-90f8-44bd-98d8-f45f2638b2e4 admin admin] PATH_INFO: `/v3/users/fadae20ed280461e806b7b7d06c7b005` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:39.298525 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f6a85-90f8-44bd-98d8-f45f2638b2e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fadae20ed280461e806b7b7d06c7b005)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:39.300746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072f6a85-90f8-44bd-98d8-f45f2638b2e4 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:39.410982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 866/1720] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:33:39 2022] POST /identity/v3/projects => generated 381 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:39.418182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94fbe5d-0490-4cb5-9089-708ab13f461e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:39.421644 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f94fbe5d-0490-4cb5-9089-708ab13f461e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:39.546155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-072f6a85-90f8-44bd-98d8-f45f2638b2e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fadae20ed280461e806b7b7d06c7b005'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:39.625379 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94fbe5d-0490-4cb5-9089-708ab13f461e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:39.632256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-072f6a85-90f8-44bd-98d8-f45f2638b2e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fadae20ed280461e806b7b7d06c7b005'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:39.650283 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5341e-b97d-48dd-97e6-4e6b31d9d848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:39.650449 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5341e-b97d-48dd-97e6-4e6b31d9d848 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:39.650875 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5341e-b97d-48dd-97e6-4e6b31d9d848 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:39.651371 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5341e-b97d-48dd-97e6-4e6b31d9d848 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:39.652089 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf5341e-b97d-48dd-97e6-4e6b31d9d848 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:39.657455 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf5341e-b97d-48dd-97e6-4e6b31d9d848 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:39.671458 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-072f6a85-90f8-44bd-98d8-f45f2638b2e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fadae20ed280461e806b7b7d06c7b005'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:39.673235 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-072f6a85-90f8-44bd-98d8-f45f2638b2e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fadae20ed280461e806b7b7d06c7b005'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:39.675161 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-072f6a85-90f8-44bd-98d8-f45f2638b2e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fadae20ed280461e806b7b7d06c7b005'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:39.732276 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 855/1721] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:33:39 2022] DELETE /identity/v3/users/fadae20ed280461e806b7b7d06c7b005 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:39.743238 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 867/1722] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:33:39 2022] POST /identity/v3/users => generated 291 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:39.748440 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f6a85-90f8-44bd-98d8-f45f2638b2e4 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:39.751625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-072f6a85-90f8-44bd-98d8-f45f2638b2e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:39.891957 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5341e-b97d-48dd-97e6-4e6b31d9d848 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:39.894644 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-dbf5341e-b97d-48dd-97e6-4e6b31d9d848 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:39.918881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5341e-b97d-48dd-97e6-4e6b31d9d848 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:39.921996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072f6a85-90f8-44bd-98d8-f45f2638b2e4 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:39.929727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6b8b92-dd64-4d86-9f19-052cd1a3ed58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:39.934544 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b8b92-dd64-4d86-9f19-052cd1a3ed58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:39.935053 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b8b92-dd64-4d86-9f19-052cd1a3ed58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:39.935490 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6b8b92-dd64-4d86-9f19-052cd1a3ed58 admin admin] PATH_INFO: `/v3/users/63d912ca14c845dc9016d6b7da2b4b5d` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:39.935743 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e2c6a-60db-45dd-841c-8c1e7a0de520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:39.936866 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e2c6a-60db-45dd-841c-8c1e7a0de520 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:39.938551 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b8b92-dd64-4d86-9f19-052cd1a3ed58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63d912ca14c845dc9016d6b7da2b4b5d)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:39.939214 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e2c6a-60db-45dd-841c-8c1e7a0de520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:39.940524 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293e2c6a-60db-45dd-841c-8c1e7a0de520 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:39.942501 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e2c6a-60db-45dd-841c-8c1e7a0de520 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:39.943624 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6b8b92-dd64-4d86-9f19-052cd1a3ed58 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:39.948433 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293e2c6a-60db-45dd-841c-8c1e7a0de520 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:39.976515 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 856/1723] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:39.978095 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e2c6a-60db-45dd-841c-8c1e7a0de520 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:40.003742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293e2c6a-60db-45dd-841c-8c1e7a0de520 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:40.055073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d1588-f763-46a8-b861-9a0c4cb64f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:40.057959 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d1588-f763-46a8-b861-9a0c4cb64f58 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:40.058345 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d1588-f763-46a8-b861-9a0c4cb64f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:40.058700 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0d1588-f763-46a8-b861-9a0c4cb64f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:40.060318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d1588-f763-46a8-b861-9a0c4cb64f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:40.068142 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0d1588-f763-46a8-b861-9a0c4cb64f58 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:40.105070 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 857/1724] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:40.106109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d1588-f763-46a8-b861-9a0c4cb64f58 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:40.126433 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0d1588-f763-46a8-b861-9a0c4cb64f58 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:40.126939 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464c6ba2-83db-4c3f-a068-4b2c019fc86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:40.127950 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c6ba2-83db-4c3f-a068-4b2c019fc86f admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:40.128245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c6ba2-83db-4c3f-a068-4b2c019fc86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:40.128529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464c6ba2-83db-4c3f-a068-4b2c019fc86f admin admin] PATH_INFO: `/v3/projects/5a3fbfe074d847d49f31b80ac458defc/users/58da31270d3740888408b972cac34354/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:40.156111 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c6ba2-83db-4c3f-a068-4b2c019fc86f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a3fbfe074d847d49f31b80ac458defc, user_id=58da31270d3740888408b972cac34354, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:40.158906 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464c6ba2-83db-4c3f-a068-4b2c019fc86f admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:40.263914 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6e6b8b92-dd64-4d86-9f19-052cd1a3ed58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63d912ca14c845dc9016d6b7da2b4b5d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:40.264846 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6e6b8b92-dd64-4d86-9f19-052cd1a3ed58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63d912ca14c845dc9016d6b7da2b4b5d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:40.273880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 858/1725] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:40 2022] PUT /identity/v3/projects/5a3fbfe074d847d49f31b80ac458defc/users/58da31270d3740888408b972cac34354/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:40.286507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66328d3f-6d15-4859-b782-2e696f4eb3e4 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:40.286808 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66328d3f-6d15-4859-b782-2e696f4eb3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:40.287076 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66328d3f-6d15-4859-b782-2e696f4eb3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:40.297926 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6e6b8b92-dd64-4d86-9f19-052cd1a3ed58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63d912ca14c845dc9016d6b7da2b4b5d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:40.297926 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6e6b8b92-dd64-4d86-9f19-052cd1a3ed58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63d912ca14c845dc9016d6b7da2b4b5d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:40.333954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-66328d3f-6d15-4859-b782-2e696f4eb3e4 None None] MFA Rules not processed for user `58da31270d3740888408b972cac34354`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:40.335237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-66328d3f-6d15-4859-b782-2e696f4eb3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:40.410550 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6e6b8b92-dd64-4d86-9f19-052cd1a3ed58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63d912ca14c845dc9016d6b7da2b4b5d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:40.441945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 868/1726] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:33:39 2022] DELETE /identity/v3/users/63d912ca14c845dc9016d6b7da2b4b5d => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:40.449164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153c6da-6008-4824-83e2-c99e84648017 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:40.449636 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153c6da-6008-4824-83e2-c99e84648017 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:40.450062 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1153c6da-6008-4824-83e2-c99e84648017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:40.496886 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1153c6da-6008-4824-83e2-c99e84648017 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:40.513633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 859/1727] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:33:40.513819 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1153c6da-6008-4824-83e2-c99e84648017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:40.513905 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66328d3f-6d15-4859-b782-2e696f4eb3e4 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:40.515787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-66328d3f-6d15-4859-b782-2e696f4eb3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:40.530027 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:40.691374 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 869/1728] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:40.695531 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153c6da-6008-4824-83e2-c99e84648017 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:40.704852 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1153c6da-6008-4824-83e2-c99e84648017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:40.736001 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66328d3f-6d15-4859-b782-2e696f4eb3e4 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:40.755817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d6032-9e5a-4e9a-9960-c56c6619fbdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:40.756722 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d6032-9e5a-4e9a-9960-c56c6619fbdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:40.757020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d6032-9e5a-4e9a-9960-c56c6619fbdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:40.757293 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149d6032-9e5a-4e9a-9960-c56c6619fbdc admin admin] PATH_INFO: `/v3/projects/473aa1f80d2644c19b829e97f7fb60d1` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:40.760050 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149d6032-9e5a-4e9a-9960-c56c6619fbdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=473aa1f80d2644c19b829e97f7fb60d1)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:40.762095 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1153c6da-6008-4824-83e2-c99e84648017 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:40.765746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149d6032-9e5a-4e9a-9960-c56c6619fbdc admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:40.785948 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd70bd-13a6-4c67-bb48-90c308f5705e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:40.786755 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd70bd-13a6-4c67-bb48-90c308f5705e admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:40.787235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd70bd-13a6-4c67-bb48-90c308f5705e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:40.792443 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdd70bd-13a6-4c67-bb48-90c308f5705e admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:40.792443 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd70bd-13a6-4c67-bb48-90c308f5705e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:40.796597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdd70bd-13a6-4c67-bb48-90c308f5705e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:40.941809 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 870/1729] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:33:40 2022] POST /identity/v3/projects => generated 381 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:40.944021 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd70bd-13a6-4c67-bb48-90c308f5705e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:40.965240 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4fdd70bd-13a6-4c67-bb48-90c308f5705e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:40.974064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-149d6032-9e5a-4e9a-9960-c56c6619fbdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '473aa1f80d2644c19b829e97f7fb60d1', 'request_id': 'req-149d6032-9e5a-4e9a-9960-c56c6619fbdc'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:41.005765 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-149d6032-9e5a-4e9a-9960-c56c6619fbdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '473aa1f80d2644c19b829e97f7fb60d1', 'request_id': 'req-149d6032-9e5a-4e9a-9960-c56c6619fbdc'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:41.021068 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-149d6032-9e5a-4e9a-9960-c56c6619fbdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '473aa1f80d2644c19b829e97f7fb60d1', 'request_id': 'req-149d6032-9e5a-4e9a-9960-c56c6619fbdc'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:41.113033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdd70bd-13a6-4c67-bb48-90c308f5705e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:41.117560 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed964fde-ccb7-4a5f-81ea-dc72067d5d40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:41.118244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed964fde-ccb7-4a5f-81ea-dc72067d5d40 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:41.118646 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed964fde-ccb7-4a5f-81ea-dc72067d5d40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:41.119667 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed964fde-ccb7-4a5f-81ea-dc72067d5d40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:41.120841 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-149d6032-9e5a-4e9a-9960-c56c6619fbdc admin admin] The token cache is being invalidate because project 473aa1f80d2644c19b829e97f7fb60d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:33:41.122058 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-149d6032-9e5a-4e9a-9960-c56c6619fbdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:41.123419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-149d6032-9e5a-4e9a-9960-c56c6619fbdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:41.135576 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 860/1730] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:33:40 2022] DELETE /identity/v3/projects/473aa1f80d2644c19b829e97f7fb60d1 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:41.136207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d6032-9e5a-4e9a-9960-c56c6619fbdc admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:41.147190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-149d6032-9e5a-4e9a-9960-c56c6619fbdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:41.285769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149d6032-9e5a-4e9a-9960-c56c6619fbdc admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:41.344481 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed964fde-ccb7-4a5f-81ea-dc72067d5d40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:41.344481 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed964fde-ccb7-4a5f-81ea-dc72067d5d40 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:41.363690 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 871/1731] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:41.368212 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872e721-315d-4ea4-bb59-f502649eb1ae None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:41.369371 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872e721-315d-4ea4-bb59-f502649eb1ae None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:41.374135 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a872e721-315d-4ea4-bb59-f502649eb1ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:41.406837 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a872e721-315d-4ea4-bb59-f502649eb1ae None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:41.411812 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a872e721-315d-4ea4-bb59-f502649eb1ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:41.413384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d790b728-0f8c-4ca3-996d-2f901cdbade6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:41.414086 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790b728-0f8c-4ca3-996d-2f901cdbade6 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:41.414371 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790b728-0f8c-4ca3-996d-2f901cdbade6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:41.414657 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790b728-0f8c-4ca3-996d-2f901cdbade6 admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:41.415373 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790b728-0f8c-4ca3-996d-2f901cdbade6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:41.417874 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d790b728-0f8c-4ca3-996d-2f901cdbade6 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:41.431092 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 872/1732] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:41.432614 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f34787-ba0f-43a6-bd40-3e8231c2f934 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:41.432902 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f34787-ba0f-43a6-bd40-3e8231c2f934 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:41.433628 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f34787-ba0f-43a6-bd40-3e8231c2f934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:41.463856 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-87f34787-ba0f-43a6-bd40-3e8231c2f934 None None] MFA Rules not processed for user `9aa86b4d8a924a068f6b4da6dbf5c6bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:41.466855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-87f34787-ba0f-43a6-bd40-3e8231c2f934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:41.559426 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:41.559670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 873/1733] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:41.559670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7810d6-8036-4894-9683-d47361196632 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:41.559670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7810d6-8036-4894-9683-d47361196632 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:41.559670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7810d6-8036-4894-9683-d47361196632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:41.569132 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 861/1734] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:33:41 2022] POST /identity/v3/users => generated 297 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:41.570756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f538e24-cf5c-485f-abfc-150a3f74b3cb None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:41.571397 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f538e24-cf5c-485f-abfc-150a3f74b3cb None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:41.571836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f538e24-cf5c-485f-abfc-150a3f74b3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:41.621105 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-8f538e24-cf5c-485f-abfc-150a3f74b3cb None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:41.624814 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-8f538e24-cf5c-485f-abfc-150a3f74b3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:41.640008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5a7810d6-8036-4894-9683-d47361196632 None None] MFA Rules not processed for user `eecc2db7b5754624b678e51dd9a37a46`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:41.641797 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5a7810d6-8036-4894-9683-d47361196632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:41.644922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 862/1735] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:41.646590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb94be2-3425-42d3-93ef-060214fa6844 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:41.647006 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb94be2-3425-42d3-93ef-060214fa6844 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:41.650901 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb94be2-3425-42d3-93ef-060214fa6844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:41.673366 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-6cb94be2-3425-42d3-93ef-060214fa6844 None None] MFA Rules not processed for user `9aa86b4d8a924a068f6b4da6dbf5c6bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:41.674841 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-6cb94be2-3425-42d3-93ef-060214fa6844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:41.684716 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 863/1736] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:41.686387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb94be2-3425-42d3-93ef-060214fa6844 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:41.702471 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb94be2-3425-42d3-93ef-060214fa6844 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:41.710984 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4605db8-e9ad-45c0-927d-625059af8b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:41.712267 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4605db8-e9ad-45c0-927d-625059af8b2e admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:41.712775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4605db8-e9ad-45c0-927d-625059af8b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:41.713229 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4605db8-e9ad-45c0-927d-625059af8b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:41.714298 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4605db8-e9ad-45c0-927d-625059af8b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:41.717746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4605db8-e9ad-45c0-927d-625059af8b2e admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:41.734454 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 864/1737] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:41.743518 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4605db8-e9ad-45c0-927d-625059af8b2e admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:41.761690 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4605db8-e9ad-45c0-927d-625059af8b2e admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:41.767719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4dd3ea-7623-41dc-a709-c08197e01235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:41.769165 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4dd3ea-7623-41dc-a709-c08197e01235 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:41.769976 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4dd3ea-7623-41dc-a709-c08197e01235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:41.770451 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4dd3ea-7623-41dc-a709-c08197e01235 admin admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:41.771418 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4dd3ea-7623-41dc-a709-c08197e01235 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:33:41.772315 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4dd3ea-7623-41dc-a709-c08197e01235 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:41.774276 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4dd3ea-7623-41dc-a709-c08197e01235 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:41.786538 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 874/1738] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:41.795822 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 865/1739] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:33:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:33:41.796567 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7810d6-8036-4894-9683-d47361196632 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:41.797203 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4dd3ea-7623-41dc-a709-c08197e01235 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:41.799187 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5a7810d6-8036-4894-9683-d47361196632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:41.811818 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4dd3ea-7623-41dc-a709-c08197e01235 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:41.817861 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0724d202-1257-4be2-a201-f979bea04230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:41.819076 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724d202-1257-4be2-a201-f979bea04230 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:41.819656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724d202-1257-4be2-a201-f979bea04230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:41.820153 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724d202-1257-4be2-a201-f979bea04230 admin admin] PATH_INFO: `/v3/projects/e97500d3d01f4d908ac699c5347082cb/users/3dcee11d98a24c3eaa24a3af3d5330cb/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:41.839031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0724d202-1257-4be2-a201-f979bea04230 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e97500d3d01f4d908ac699c5347082cb, user_id=3dcee11d98a24c3eaa24a3af3d5330cb, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:41.842506 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0724d202-1257-4be2-a201-f979bea04230 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:41.914322 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 866/1740] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:41 2022] PUT /identity/v3/projects/e97500d3d01f4d908ac699c5347082cb/users/3dcee11d98a24c3eaa24a3af3d5330cb/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:41.917207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe377a-3e55-4184-b75d-f68d2108a988 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:41.917512 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe377a-3e55-4184-b75d-f68d2108a988 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:41.917792 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe377a-3e55-4184-b75d-f68d2108a988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:41.973920 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7810d6-8036-4894-9683-d47361196632 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:41.984926 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-aafe377a-3e55-4184-b75d-f68d2108a988 None None] MFA Rules not processed for user `eecc2db7b5754624b678e51dd9a37a46`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:41.986197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-aafe377a-3e55-4184-b75d-f68d2108a988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:42.058473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:42.133689 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 867/1741] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:42.137943 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe377a-3e55-4184-b75d-f68d2108a988 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:42.144491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30734710-fe58-41f9-a3a1-f73417778c7e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:42.145364 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30734710-fe58-41f9-a3a1-f73417778c7e service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:42.145780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30734710-fe58-41f9-a3a1-f73417778c7e service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:42.146019 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30734710-fe58-41f9-a3a1-f73417778c7e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:42.258574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe377a-3e55-4184-b75d-f68d2108a988 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:42.265423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a241966-0856-4126-93d2-dd24a6e9533d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:42.266297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a241966-0856-4126-93d2-dd24a6e9533d admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:42.266680 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a241966-0856-4126-93d2-dd24a6e9533d admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:42.267102 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a241966-0856-4126-93d2-dd24a6e9533d admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:42.268600 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a241966-0856-4126-93d2-dd24a6e9533d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:42.269999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30734710-fe58-41f9-a3a1-f73417778c7e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:42.271277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a241966-0856-4126-93d2-dd24a6e9533d admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:42.272299 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30734710-fe58-41f9-a3a1-f73417778c7e service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:42.285888 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 875/1742] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:42.287065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30734710-fe58-41f9-a3a1-f73417778c7e service nova] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:42.303953 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30734710-fe58-41f9-a3a1-f73417778c7e service nova] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:42.311940 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d6040-3ad5-4fe2-831a-f37df21a1d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:42.312800 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d6040-3ad5-4fe2-831a-f37df21a1d7a admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:42.313214 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d6040-3ad5-4fe2-831a-f37df21a1d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:42.313495 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48d6040-3ad5-4fe2-831a-f37df21a1d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:42.314247 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48d6040-3ad5-4fe2-831a-f37df21a1d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:42.316746 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48d6040-3ad5-4fe2-831a-f37df21a1d7a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:42.333298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 876/1743] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:42.334350 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d6040-3ad5-4fe2-831a-f37df21a1d7a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:42.336755 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e48d6040-3ad5-4fe2-831a-f37df21a1d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:42.360125 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48d6040-3ad5-4fe2-831a-f37df21a1d7a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:42.373970 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44c5c5-d257-415b-88ea-60e1e0d39746 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:42.374808 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44c5c5-d257-415b-88ea-60e1e0d39746 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:42.375239 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44c5c5-d257-415b-88ea-60e1e0d39746 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:42.375574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f44c5c5-d257-415b-88ea-60e1e0d39746 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:42.386734 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f44c5c5-d257-415b-88ea-60e1e0d39746 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:42.388848 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f44c5c5-d257-415b-88ea-60e1e0d39746 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:42.401516 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 877/1744] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:42.402402 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44c5c5-d257-415b-88ea-60e1e0d39746 service nova] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:42.403899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7f44c5c5-d257-415b-88ea-60e1e0d39746 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:42.422005 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f44c5c5-d257-415b-88ea-60e1e0d39746 service nova] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:42.430681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 868/1745] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:33:42 2022] POST /identity/v3/projects => generated 371 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:42.433562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a241966-0856-4126-93d2-dd24a6e9533d admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:42.438538 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36131c2e-a3f8-4817-846d-ecbad6c6aa17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:42.438994 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36131c2e-a3f8-4817-846d-ecbad6c6aa17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:42.439463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36131c2e-a3f8-4817-846d-ecbad6c6aa17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:42.439943 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36131c2e-a3f8-4817-846d-ecbad6c6aa17 admin admin] PATH_INFO: `/v3/projects/87089848a6ea4d579579f1d3ebbd8965` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:42.443178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36131c2e-a3f8-4817-846d-ecbad6c6aa17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87089848a6ea4d579579f1d3ebbd8965)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:42.445802 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36131c2e-a3f8-4817-846d-ecbad6c6aa17 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:42.625984 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-36131c2e-a3f8-4817-846d-ecbad6c6aa17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87089848a6ea4d579579f1d3ebbd8965', 'request_id': 'req-36131c2e-a3f8-4817-846d-ecbad6c6aa17'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:42.631488 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a241966-0856-4126-93d2-dd24a6e9533d admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:42.706742 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-36131c2e-a3f8-4817-846d-ecbad6c6aa17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87089848a6ea4d579579f1d3ebbd8965', 'request_id': 'req-36131c2e-a3f8-4817-846d-ecbad6c6aa17'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:42.716787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-36131c2e-a3f8-4817-846d-ecbad6c6aa17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87089848a6ea4d579579f1d3ebbd8965', 'request_id': 'req-36131c2e-a3f8-4817-846d-ecbad6c6aa17'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:42.774509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-36131c2e-a3f8-4817-846d-ecbad6c6aa17 admin admin] The token cache is being invalidate because project 87089848a6ea4d579579f1d3ebbd8965 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:33:42.775021 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-36131c2e-a3f8-4817-846d-ecbad6c6aa17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:42.779965 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-36131c2e-a3f8-4817-846d-ecbad6c6aa17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:42.779965 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 878/1746] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:33:42 2022] DELETE /identity/v3/projects/87089848a6ea4d579579f1d3ebbd8965 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:42.781671 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32809798-8193-4fa1-9147-557b0a9a46bc None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:42.782144 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32809798-8193-4fa1-9147-557b0a9a46bc None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:42.783609 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32809798-8193-4fa1-9147-557b0a9a46bc None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:42.786120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 879/1747] 10.222.0.47 () {54 vars in 912 bytes} [Thu Jun 30 10:33:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:33:42.787340 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32809798-8193-4fa1-9147-557b0a9a46bc None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:42.802710 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-32809798-8193-4fa1-9147-557b0a9a46bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:42.802890 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:42.808227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3956f-bc8e-4885-8f0f-26e771bac63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:42.809466 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3956f-bc8e-4885-8f0f-26e771bac63e admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:42.809956 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3956f-bc8e-4885-8f0f-26e771bac63e admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:42.810430 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3956f-bc8e-4885-8f0f-26e771bac63e admin admin] PATH_INFO: `/v3/projects/e97500d3d01f4d908ac699c5347082cb/users/3dcee11d98a24c3eaa24a3af3d5330cb/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:42.818915 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3956f-bc8e-4885-8f0f-26e771bac63e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e97500d3d01f4d908ac699c5347082cb, user_id=3dcee11d98a24c3eaa24a3af3d5330cb, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:42.823946 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3956f-bc8e-4885-8f0f-26e771bac63e admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:42.848559 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 869/1748] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:42 2022] PUT /identity/v3/projects/e97500d3d01f4d908ac699c5347082cb/users/3dcee11d98a24c3eaa24a3af3d5330cb/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:42.850211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e79ce0-1f5e-4d05-aaed-744cecf30cbf None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:42.850609 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e79ce0-1f5e-4d05-aaed-744cecf30cbf None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:42.850990 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e79ce0-1f5e-4d05-aaed-744cecf30cbf None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:42.852725 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 870/1749] 10.222.0.47 () {54 vars in 912 bytes} [Thu Jun 30 10:33:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:33:42.854036 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e79ce0-1f5e-4d05-aaed-744cecf30cbf None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:42.856351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-b4e79ce0-1f5e-4d05-aaed-744cecf30cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:42.878244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32809798-8193-4fa1-9147-557b0a9a46bc None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:43.026087 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e79ce0-1f5e-4d05-aaed-744cecf30cbf None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:43.040283 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc66caf-e9e6-4f59-922b-3be12e6c091d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:43.041839 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f4f39-e63c-4873-aa0b-7b94bf846c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:43.042733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc66caf-e9e6-4f59-922b-3be12e6c091d admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:43.043122 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc66caf-e9e6-4f59-922b-3be12e6c091d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:43.043833 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f4f39-e63c-4873-aa0b-7b94bf846c5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:43.045521 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc66caf-e9e6-4f59-922b-3be12e6c091d admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:43.047223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f4f39-e63c-4873-aa0b-7b94bf846c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:43.047817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f4f39-e63c-4873-aa0b-7b94bf846c5e admin admin] PATH_INFO: `/v3/users/f0e1c6cb69064c409f72b10f9dfc3a79` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:43.048068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc66caf-e9e6-4f59-922b-3be12e6c091d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:43.050525 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f4f39-e63c-4873-aa0b-7b94bf846c5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0e1c6cb69064c409f72b10f9dfc3a79)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:43.054748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc66caf-e9e6-4f59-922b-3be12e6c091d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:43.056227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f4f39-e63c-4873-aa0b-7b94bf846c5e admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:43.170403 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 880/1750] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:33:42 2022] POST /identity/v3/users => generated 286 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:43.179687 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004dcf2c-9d5b-4dc6-94ec-cb3279d2fa98 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:43.180209 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004dcf2c-9d5b-4dc6-94ec-cb3279d2fa98 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:43.183974 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004dcf2c-9d5b-4dc6-94ec-cb3279d2fa98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:43.223248 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-004dcf2c-9d5b-4dc6-94ec-cb3279d2fa98 None None] MFA Rules not processed for user `71f6d7715e6f49ffb9fdf6e1307bbd51`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:43.224655 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-004dcf2c-9d5b-4dc6-94ec-cb3279d2fa98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:43.303624 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 881/1751] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jun 30 10:33:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:43.308060 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bac41d-82a8-4731-9f86-44bf2657abb8 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:43.308631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bac41d-82a8-4731-9f86-44bf2657abb8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:43.332208 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bac41d-82a8-4731-9f86-44bf2657abb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:43.335453 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-039f4f39-e63c-4873-aa0b-7b94bf846c5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0e1c6cb69064c409f72b10f9dfc3a79'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:43.386675 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-25bac41d-82a8-4731-9f86-44bf2657abb8 None None] MFA Rules not processed for user `3dcee11d98a24c3eaa24a3af3d5330cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:43.389688 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-25bac41d-82a8-4731-9f86-44bf2657abb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:43.471917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-039f4f39-e63c-4873-aa0b-7b94bf846c5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0e1c6cb69064c409f72b10f9dfc3a79'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:43.535478 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-039f4f39-e63c-4873-aa0b-7b94bf846c5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0e1c6cb69064c409f72b10f9dfc3a79'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:43.537329 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-039f4f39-e63c-4873-aa0b-7b94bf846c5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0e1c6cb69064c409f72b10f9dfc3a79'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:43.539680 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-039f4f39-e63c-4873-aa0b-7b94bf846c5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0e1c6cb69064c409f72b10f9dfc3a79'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:43.585253 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 871/1752] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:33:42 2022] DELETE /identity/v3/users/f0e1c6cb69064c409f72b10f9dfc3a79 => generated 0 bytes in 729 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:43.585253 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e115641-a9f2-4252-bd51-7565a9d0bacd None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:43.585253 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e115641-a9f2-4252-bd51-7565a9d0bacd None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:43.585253 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e115641-a9f2-4252-bd51-7565a9d0bacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:43.590409 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 882/1753] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:43 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:43.591319 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bac41d-82a8-4731-9f86-44bf2657abb8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:43.595737 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-25bac41d-82a8-4731-9f86-44bf2657abb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:43.646241 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-5e115641-a9f2-4252-bd51-7565a9d0bacd None None] MFA Rules not processed for user `71f6d7715e6f49ffb9fdf6e1307bbd51`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:43.646241 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-5e115641-a9f2-4252-bd51-7565a9d0bacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:43.653756 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=87,keystone:INSERT=5,keystone:DELETE=8 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:43.739500 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bac41d-82a8-4731-9f86-44bf2657abb8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:43.837021 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 872/1754] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jun 30 10:33:43 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:43.838969 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e115641-a9f2-4252-bd51-7565a9d0bacd None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:43.842359 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-5e115641-a9f2-4252-bd51-7565a9d0bacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:43.909058 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19e8fe-701b-4a87-8927-56d162941ecb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:43.909740 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19e8fe-701b-4a87-8927-56d162941ecb service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:43.910023 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19e8fe-701b-4a87-8927-56d162941ecb service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:43.910305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f19e8fe-701b-4a87-8927-56d162941ecb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:43.912325 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8f19e8fe-701b-4a87-8927-56d162941ecb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:43.939877 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e115641-a9f2-4252-bd51-7565a9d0bacd None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:43.945602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042962d7-8558-432d-844d-c2bc0b0778d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:43.947326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042962d7-8558-432d-844d-c2bc0b0778d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:43.950234 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042962d7-8558-432d-844d-c2bc0b0778d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:43.950234 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042962d7-8558-432d-844d-c2bc0b0778d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:43.950234 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042962d7-8558-432d-844d-c2bc0b0778d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:43.951437 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042962d7-8558-432d-844d-c2bc0b0778d0 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:43.964980 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 873/1755] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:43 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:43.964980 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042962d7-8558-432d-844d-c2bc0b0778d0 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:43.967808 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-042962d7-8558-432d-844d-c2bc0b0778d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:44.055347 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042962d7-8558-432d-844d-c2bc0b0778d0 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:44.059087 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f19e8fe-701b-4a87-8927-56d162941ecb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:44.061427 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f19e8fe-701b-4a87-8927-56d162941ecb service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:44.069628 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c827c0-ab24-492e-9fef-59d996c6f7a3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7a61d85908d42698b27b9fc00daa832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:44.070356 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c827c0-ab24-492e-9fef-59d996c6f7a3 service glance] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:44.070682 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c827c0-ab24-492e-9fef-59d996c6f7a3 service glance] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:44.070968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c827c0-ab24-492e-9fef-59d996c6f7a3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:44.077197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-c8c827c0-ab24-492e-9fef-59d996c6f7a3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:44.090442 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 883/1756] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:43 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:44.091461 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19e8fe-701b-4a87-8927-56d162941ecb service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:44.095489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8f19e8fe-701b-4a87-8927-56d162941ecb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:44.112886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c827c0-ab24-492e-9fef-59d996c6f7a3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:44.116951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c827c0-ab24-492e-9fef-59d996c6f7a3 service glance] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:44.129445 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f19e8fe-701b-4a87-8927-56d162941ecb service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:44.142765 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efdf72-1b28-4330-9e9b-df579d26e4dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:44.144628 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efdf72-1b28-4330-9e9b-df579d26e4dd service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:44.144918 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efdf72-1b28-4330-9e9b-df579d26e4dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:44.146409 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15efdf72-1b28-4330-9e9b-df579d26e4dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:44.151868 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 874/1757] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:44.153460 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c827c0-ab24-492e-9fef-59d996c6f7a3 service glance] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:44.157481 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-c8c827c0-ab24-492e-9fef-59d996c6f7a3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:44.194800 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c827c0-ab24-492e-9fef-59d996c6f7a3 service glance] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:44.211961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d827fd-eade-44bc-a09a-e9e791ac28a3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7a61d85908d42698b27b9fc00daa832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:44.214221 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d827fd-eade-44bc-a09a-e9e791ac28a3 service glance] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:44.215749 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d827fd-eade-44bc-a09a-e9e791ac28a3 service glance] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:44.216368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d827fd-eade-44bc-a09a-e9e791ac28a3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:44.227444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:44.254528 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d827fd-eade-44bc-a09a-e9e791ac28a3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:44.256386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d827fd-eade-44bc-a09a-e9e791ac28a3 service glance] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:44.287347 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 875/1758] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:44.288480 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d827fd-eade-44bc-a09a-e9e791ac28a3 service glance] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:44.294772 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efdf72-1b28-4330-9e9b-df579d26e4dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:44.296595 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15efdf72-1b28-4330-9e9b-df579d26e4dd service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:44.312214 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 884/1759] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:44 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:44.313502 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d827fd-eade-44bc-a09a-e9e791ac28a3 service glance] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:44.332544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec45b1-ab6a-4da5-b2b4-2ece6462ad2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:44.341079 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec45b1-ab6a-4da5-b2b4-2ece6462ad2f admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:44.341673 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec45b1-ab6a-4da5-b2b4-2ece6462ad2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:44.341673 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ec45b1-ab6a-4da5-b2b4-2ece6462ad2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:44.342428 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec45b1-ab6a-4da5-b2b4-2ece6462ad2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:44.360099 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ec45b1-ab6a-4da5-b2b4-2ece6462ad2f admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:44.403434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 876/1760] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:44 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:44.403589 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efdf72-1b28-4330-9e9b-df579d26e4dd service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:44.424242 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15efdf72-1b28-4330-9e9b-df579d26e4dd service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:44.430311 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec45b1-ab6a-4da5-b2b4-2ece6462ad2f admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:44.434002 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-52ec45b1-ab6a-4da5-b2b4-2ece6462ad2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:44.435569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec9805d-365c-4f90-8534-5ba918ecbe61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:44.443540 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9805d-365c-4f90-8534-5ba918ecbe61 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:44.443769 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9805d-365c-4f90-8534-5ba918ecbe61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:44.444059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec9805d-365c-4f90-8534-5ba918ecbe61 admin admin] PATH_INFO: `/v3/projects/d717eb7df90542af8edd4610e7e4d4d0/users/22efe134e4534ee78456833b4b690f88/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:44.480053 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec9805d-365c-4f90-8534-5ba918ecbe61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d717eb7df90542af8edd4610e7e4d4d0, user_id=22efe134e4534ee78456833b4b690f88, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:44.486222 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec9805d-365c-4f90-8534-5ba918ecbe61 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:44.511850 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ec45b1-ab6a-4da5-b2b4-2ece6462ad2f admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:44.534938 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ed52bb-fc6f-41dd-b3ab-5d79fa8ec170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:44.535935 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed52bb-fc6f-41dd-b3ab-5d79fa8ec170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:44.537462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed52bb-fc6f-41dd-b3ab-5d79fa8ec170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:44.539645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ed52bb-fc6f-41dd-b3ab-5d79fa8ec170 admin admin] PATH_INFO: `/v3/projects/13a33103e5fb468185777067be2d2240` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:44.541074 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed52bb-fc6f-41dd-b3ab-5d79fa8ec170 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13a33103e5fb468185777067be2d2240)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:44.547511 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ed52bb-fc6f-41dd-b3ab-5d79fa8ec170 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:44.623240 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 885/1761] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:44 2022] PUT /identity/v3/projects/d717eb7df90542af8edd4610e7e4d4d0/users/22efe134e4534ee78456833b4b690f88/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:44.625502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92987d23-beed-46c1-86af-d36a6e1dd426 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:44.625665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92987d23-beed-46c1-86af-d36a6e1dd426 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:44.626509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92987d23-beed-46c1-86af-d36a6e1dd426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:44.702291 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-92987d23-beed-46c1-86af-d36a6e1dd426 None None] MFA Rules not processed for user `22efe134e4534ee78456833b4b690f88`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:44.708958 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-92987d23-beed-46c1-86af-d36a6e1dd426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:44.794000 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-59ed52bb-fc6f-41dd-b3ab-5d79fa8ec170 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13a33103e5fb468185777067be2d2240', 'request_id': 'req-59ed52bb-fc6f-41dd-b3ab-5d79fa8ec170'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:44.821243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-59ed52bb-fc6f-41dd-b3ab-5d79fa8ec170 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13a33103e5fb468185777067be2d2240', 'request_id': 'req-59ed52bb-fc6f-41dd-b3ab-5d79fa8ec170'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:44.844262 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-59ed52bb-fc6f-41dd-b3ab-5d79fa8ec170 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13a33103e5fb468185777067be2d2240', 'request_id': 'req-59ed52bb-fc6f-41dd-b3ab-5d79fa8ec170'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:44.907325 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 886/1762] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:44.915671 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92987d23-beed-46c1-86af-d36a6e1dd426 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:44.936458 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-59ed52bb-fc6f-41dd-b3ab-5d79fa8ec170 admin admin] The token cache is being invalidate because project 13a33103e5fb468185777067be2d2240 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:33:44.936837 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-59ed52bb-fc6f-41dd-b3ab-5d79fa8ec170 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:44.939757 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-59ed52bb-fc6f-41dd-b3ab-5d79fa8ec170 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:44.942318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 877/1763] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:33:44 2022] DELETE /identity/v3/projects/13a33103e5fb468185777067be2d2240 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:44.948902 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f9ad5-56a6-4bfa-aa39-b1a14a2627ce None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:44.949202 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f9ad5-56a6-4bfa-aa39-b1a14a2627ce None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:44.949483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099f9ad5-56a6-4bfa-aa39-b1a14a2627ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:45.025756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-099f9ad5-56a6-4bfa-aa39-b1a14a2627ce None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:45.029403 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-099f9ad5-56a6-4bfa-aa39-b1a14a2627ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:45.100829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92987d23-beed-46c1-86af-d36a6e1dd426 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:45.125263 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 878/1764] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:33:45.127015 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39669153-f2cd-4f3b-8a64-64f4ebd1e0a6 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:45.127338 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39669153-f2cd-4f3b-8a64-64f4ebd1e0a6 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:45.127622 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39669153-f2cd-4f3b-8a64-64f4ebd1e0a6 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:45.129347 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 879/1765] 10.222.0.47 () {54 vars in 912 bytes} [Thu Jun 30 10:33:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:33:45.134381 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90d03e-d889-4386-99e4-69b003557eaa None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:45.134778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90d03e-d889-4386-99e4-69b003557eaa None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:45.135296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90d03e-d889-4386-99e4-69b003557eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:45.179019 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-5a90d03e-d889-4386-99e4-69b003557eaa None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:45.182289 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-5a90d03e-d889-4386-99e4-69b003557eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:45.202651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 880/1766] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:45.206398 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441874c-f59b-4578-92f1-18fd67dbfb5d None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:45.206697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441874c-f59b-4578-92f1-18fd67dbfb5d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:45.206976 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441874c-f59b-4578-92f1-18fd67dbfb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:45.288161 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec5cfa3-3ee0-44cb-8f46-f5780ee8aa5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:45.289112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5cfa3-3ee0-44cb-8f46-f5780ee8aa5b admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:45.289599 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5cfa3-3ee0-44cb-8f46-f5780ee8aa5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:45.290889 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec5cfa3-3ee0-44cb-8f46-f5780ee8aa5b admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:45.292621 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec5cfa3-3ee0-44cb-8f46-f5780ee8aa5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:45.295403 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec5cfa3-3ee0-44cb-8f46-f5780ee8aa5b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:45.318965 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-5441874c-f59b-4578-92f1-18fd67dbfb5d None None] MFA Rules not processed for user `059b05007c0144f7a50dbd9f0ef41359`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:45.320235 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-5441874c-f59b-4578-92f1-18fd67dbfb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:45.436798 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 887/1767] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:33:44 2022] POST /identity/v3/projects => generated 371 bytes in 520 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:45.443533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 881/1768] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jun 30 10:33:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:45.445191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44896a-4c77-4bef-8461-8932af4b8ff1 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:45.445546 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44896a-4c77-4bef-8461-8932af4b8ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:45.445889 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c44896a-4c77-4bef-8461-8932af4b8ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:45.449498 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec5cfa3-3ee0-44cb-8f46-f5780ee8aa5b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:45.458022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:45.468276 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-7c44896a-4c77-4bef-8461-8932af4b8ff1 None None] MFA Rules not processed for user `797a7b1e128f4102bac6430a60d2c0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:45.470256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-7c44896a-4c77-4bef-8461-8932af4b8ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:45.553958 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:45.582299 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec5cfa3-3ee0-44cb-8f46-f5780ee8aa5b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:45.590488 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d6ce9-ac8a-4f72-bfcc-72e08b7a7a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:45.591722 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d6ce9-ac8a-4f72-bfcc-72e08b7a7a95 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:45.592698 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d6ce9-ac8a-4f72-bfcc-72e08b7a7a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:45.593540 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d6ce9-ac8a-4f72-bfcc-72e08b7a7a95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:45.594504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d6ce9-ac8a-4f72-bfcc-72e08b7a7a95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:33:45.595044 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d6ce9-ac8a-4f72-bfcc-72e08b7a7a95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:45.597631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d6ce9-ac8a-4f72-bfcc-72e08b7a7a95 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:45.610552 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 882/1769] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:45.614419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44896a-4c77-4bef-8461-8932af4b8ff1 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:45.614419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-7c44896a-4c77-4bef-8461-8932af4b8ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:45.636724 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c44896a-4c77-4bef-8461-8932af4b8ff1 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:45.639112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 888/1770] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:33:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:33:45.640941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d6ce9-ac8a-4f72-bfcc-72e08b7a7a95 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:45.645913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-462d6ce9-ac8a-4f72-bfcc-72e08b7a7a95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:45.648395 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c37a83-c0df-4f67-b825-843665fbabbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:45.649091 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c37a83-c0df-4f67-b825-843665fbabbc admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:45.649378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c37a83-c0df-4f67-b825-843665fbabbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:45.650732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c37a83-c0df-4f67-b825-843665fbabbc admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:45.650936 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c37a83-c0df-4f67-b825-843665fbabbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:45.653068 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c37a83-c0df-4f67-b825-843665fbabbc admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:45.733414 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d6ce9-ac8a-4f72-bfcc-72e08b7a7a95 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:45.737239 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c2381-fc5b-4445-a696-5a880cf91f9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:45.737903 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c2381-fc5b-4445-a696-5a880cf91f9b service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:45.738194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c2381-fc5b-4445-a696-5a880cf91f9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:45.738502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c2381-fc5b-4445-a696-5a880cf91f9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:45.747129 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 883/1771] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:33:45 2022] POST /identity/v3/users => generated 292 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:45.748167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c37a83-c0df-4f67-b825-843665fbabbc admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:45.748440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c2381-fc5b-4445-a696-5a880cf91f9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:45.749937 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-00c37a83-c0df-4f67-b825-843665fbabbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:45.750120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c2381-fc5b-4445-a696-5a880cf91f9b service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:45.760256 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 889/1772] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:45.761469 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c2381-fc5b-4445-a696-5a880cf91f9b service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:45.773646 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c2381-fc5b-4445-a696-5a880cf91f9b service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:45.778438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9394d845-d6b2-4f50-a13c-d1a74d5c1cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:45.780833 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394d845-d6b2-4f50-a13c-d1a74d5c1cc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:45.780833 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394d845-d6b2-4f50-a13c-d1a74d5c1cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:45.780833 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9394d845-d6b2-4f50-a13c-d1a74d5c1cc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:45.780833 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394d845-d6b2-4f50-a13c-d1a74d5c1cc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:45.782628 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9394d845-d6b2-4f50-a13c-d1a74d5c1cc9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:45.784203 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c37a83-c0df-4f67-b825-843665fbabbc admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:45.800134 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0efa3a2-4f40-4596-8e3a-8e384047fbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:45.801179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efa3a2-4f40-4596-8e3a-8e384047fbd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:45.801521 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efa3a2-4f40-4596-8e3a-8e384047fbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:45.801818 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0efa3a2-4f40-4596-8e3a-8e384047fbd2 admin admin] PATH_INFO: `/v3/users/58da31270d3740888408b972cac34354` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:45.803255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0efa3a2-4f40-4596-8e3a-8e384047fbd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58da31270d3740888408b972cac34354)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:45.805404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0efa3a2-4f40-4596-8e3a-8e384047fbd2 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:45.905922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 890/1773] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:33:45 2022] POST /identity/v3/projects => generated 355 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:45.907515 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4ea36-4a55-42e2-b02a-44be3f808c55 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:45.907814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4ea36-4a55-42e2-b02a-44be3f808c55 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:45.908099 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b4ea36-4a55-42e2-b02a-44be3f808c55 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:45.909921 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 891/1774] 10.222.0.47 () {54 vars in 912 bytes} [Thu Jun 30 10:33:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:33:45.915115 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4ea36-4a55-42e2-b02a-44be3f808c55 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:46.012198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e0efa3a2-4f40-4596-8e3a-8e384047fbd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58da31270d3740888408b972cac34354'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:46.014575 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b4ea36-4a55-42e2-b02a-44be3f808c55 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:46.022131 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e57126-efe2-4eb1-aa71-02c944185082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:46.022131 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e57126-efe2-4eb1-aa71-02c944185082 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:46.022659 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e57126-efe2-4eb1-aa71-02c944185082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:46.022659 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e57126-efe2-4eb1-aa71-02c944185082 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:46.023354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e57126-efe2-4eb1-aa71-02c944185082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:46.025687 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e57126-efe2-4eb1-aa71-02c944185082 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:46.038151 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 892/1775] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:45 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:46.041157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7acf23-c01f-45ab-922d-cb8ab11570b9 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:46.041157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7acf23-c01f-45ab-922d-cb8ab11570b9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:46.041157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7acf23-c01f-45ab-922d-cb8ab11570b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:46.064902 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6e7acf23-c01f-45ab-922d-cb8ab11570b9 None None] MFA Rules not processed for user `c7f8abef711c4cd2b53852b3a969b120`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:46.066729 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6e7acf23-c01f-45ab-922d-cb8ab11570b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:46.100171 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e0efa3a2-4f40-4596-8e3a-8e384047fbd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58da31270d3740888408b972cac34354'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:46.119573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e0efa3a2-4f40-4596-8e3a-8e384047fbd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58da31270d3740888408b972cac34354'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:46.120407 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e0efa3a2-4f40-4596-8e3a-8e384047fbd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58da31270d3740888408b972cac34354'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:46.122281 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e0efa3a2-4f40-4596-8e3a-8e384047fbd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58da31270d3740888408b972cac34354'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:46.146431 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 884/1776] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:33:45 2022] DELETE /identity/v3/users/58da31270d3740888408b972cac34354 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:46.147040 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0efa3a2-4f40-4596-8e3a-8e384047fbd2 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:46.148437 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-e0efa3a2-4f40-4596-8e3a-8e384047fbd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:46.164027 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 893/1777] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:46.169330 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ce64a-d263-4f92-aa06-b26082bd977e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:46.169330 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ce64a-d263-4f92-aa06-b26082bd977e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:46.169330 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ce64a-d263-4f92-aa06-b26082bd977e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:46.187811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b27ce64a-d263-4f92-aa06-b26082bd977e None None] MFA Rules not processed for user `059b05007c0144f7a50dbd9f0ef41359`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:46.189172 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b27ce64a-d263-4f92-aa06-b26082bd977e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:46.235367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0efa3a2-4f40-4596-8e3a-8e384047fbd2 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:46.244782 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6e0f5-cc8d-48bb-b155-023f5020c02a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:46.245654 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6e0f5-cc8d-48bb-b155-023f5020c02a admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:46.246050 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6e0f5-cc8d-48bb-b155-023f5020c02a admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:46.246570 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6e0f5-cc8d-48bb-b155-023f5020c02a admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:46.247253 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6e0f5-cc8d-48bb-b155-023f5020c02a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:46.249405 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6e0f5-cc8d-48bb-b155-023f5020c02a admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:46.262450 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 894/1778] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jun 30 10:33:46 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:46.263427 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ce64a-d263-4f92-aa06-b26082bd977e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:46.265022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b27ce64a-d263-4f92-aa06-b26082bd977e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:46.283033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ce64a-d263-4f92-aa06-b26082bd977e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:46.288113 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50891f05-922f-4565-834f-0d9a6209087f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:46.288963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50891f05-922f-4565-834f-0d9a6209087f admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:46.289245 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50891f05-922f-4565-834f-0d9a6209087f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:46.289520 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50891f05-922f-4565-834f-0d9a6209087f admin admin] PATH_INFO: `/v3/projects/07d3e67c87d945b7aa3d135625239925/users/10e7373619ed483db3030760aa0e695e/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:46.305687 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50891f05-922f-4565-834f-0d9a6209087f admin admin] RBAC: Authorizing `identity:create_grant(project_id=07d3e67c87d945b7aa3d135625239925, user_id=10e7373619ed483db3030760aa0e695e, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:46.308503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50891f05-922f-4565-834f-0d9a6209087f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:46.358081 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 885/1779] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:33:46 2022] POST /identity/v3/users => generated 278 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:46.358081 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe71761-bf4c-4f60-b4e2-1bb4b49a5f02 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:46.358081 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe71761-bf4c-4f60-b4e2-1bb4b49a5f02 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:46.358081 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe71761-bf4c-4f60-b4e2-1bb4b49a5f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:46.386402 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 895/1780] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:46 2022] PUT /identity/v3/projects/07d3e67c87d945b7aa3d135625239925/users/10e7373619ed483db3030760aa0e695e/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:46.387522 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50891f05-922f-4565-834f-0d9a6209087f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:46.396573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-50891f05-922f-4565-834f-0d9a6209087f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:46.433599 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-1fe71761-bf4c-4f60-b4e2-1bb4b49a5f02 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:46.435120 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-1fe71761-bf4c-4f60-b4e2-1bb4b49a5f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:46.468892 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50891f05-922f-4565-834f-0d9a6209087f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:46.472939 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b295e-9752-46db-b929-50dba641977b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:46.473683 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b295e-9752-46db-b929-50dba641977b service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:46.473972 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b295e-9752-46db-b929-50dba641977b service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:46.474249 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2b295e-9752-46db-b929-50dba641977b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:46.483393 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b295e-9752-46db-b929-50dba641977b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:46.488680 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2b295e-9752-46db-b929-50dba641977b service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:46.496661 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 896/1781] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:46.497734 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b295e-9752-46db-b929-50dba641977b service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:46.656012 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 886/1782] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:46.663630 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe71761-bf4c-4f60-b4e2-1bb4b49a5f02 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:46.677195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-1fe71761-bf4c-4f60-b4e2-1bb4b49a5f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:46.692034 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2b295e-9752-46db-b929-50dba641977b service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:46.708381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb635cb4-c28f-41ba-a9f2-8933580892ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:46.709066 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb635cb4-c28f-41ba-a9f2-8933580892ee admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:46.709352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb635cb4-c28f-41ba-a9f2-8933580892ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:46.709630 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb635cb4-c28f-41ba-a9f2-8933580892ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:46.710364 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb635cb4-c28f-41ba-a9f2-8933580892ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:46.712027 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb635cb4-c28f-41ba-a9f2-8933580892ee admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:46.744917 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 897/1783] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:46.745852 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb635cb4-c28f-41ba-a9f2-8933580892ee admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:46.779511 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb635cb4-c28f-41ba-a9f2-8933580892ee admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:46.793042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b91a7-0498-4c10-93a2-ab27328b8a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:46.793731 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b91a7-0498-4c10-93a2-ab27328b8a4e admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:46.794014 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b91a7-0498-4c10-93a2-ab27328b8a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:46.794289 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b91a7-0498-4c10-93a2-ab27328b8a4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:46.795071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b91a7-0498-4c10-93a2-ab27328b8a4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:46.810774 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b91a7-0498-4c10-93a2-ab27328b8a4e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:46.823379 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe71761-bf4c-4f60-b4e2-1bb4b49a5f02 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:46.830040 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a977c-7c76-49e9-9f96-8cf45164a176 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46c99f43a9344d7d92632990e45bb9cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:46.830867 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a977c-7c76-49e9-9f96-8cf45164a176 service placement] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:46.831164 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a977c-7c76-49e9-9f96-8cf45164a176 service placement] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:46.831504 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289a977c-7c76-49e9-9f96-8cf45164a176 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:46.831742 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 898/1784] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:46.834187 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-289a977c-7c76-49e9-9f96-8cf45164a176 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:46.835755 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b91a7-0498-4c10-93a2-ab27328b8a4e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:46.845725 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4e6b91a7-0498-4c10-93a2-ab27328b8a4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:46.848181 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289a977c-7c76-49e9-9f96-8cf45164a176 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:46.849866 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289a977c-7c76-49e9-9f96-8cf45164a176 service placement] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:46.860826 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b91a7-0498-4c10-93a2-ab27328b8a4e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:46.865853 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4696a-f614-451c-a22f-76fad727ed28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:46.866226 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 887/1785] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:46.866899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4696a-f614-451c-a22f-76fad727ed28 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:46.867212 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4696a-f614-451c-a22f-76fad727ed28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:46.867525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4696a-f614-451c-a22f-76fad727ed28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:46.881296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a977c-7c76-49e9-9f96-8cf45164a176 service placement] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:46.894192 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb4696a-f614-451c-a22f-76fad727ed28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:46.895700 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb4696a-f614-451c-a22f-76fad727ed28 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:46.912276 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289a977c-7c76-49e9-9f96-8cf45164a176 service placement] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:46.918218 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880908fa-2ea2-4581-afc9-5e2ecbc7a504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:46.918913 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880908fa-2ea2-4581-afc9-5e2ecbc7a504 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:46.919206 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880908fa-2ea2-4581-afc9-5e2ecbc7a504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:46.919564 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880908fa-2ea2-4581-afc9-5e2ecbc7a504 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:46.920313 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880908fa-2ea2-4581-afc9-5e2ecbc7a504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:46.921366 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 899/1786] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:46.922319 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880908fa-2ea2-4581-afc9-5e2ecbc7a504 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:46.923812 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4696a-f614-451c-a22f-76fad727ed28 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:46.939449 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 888/1787] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:46 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:46.940392 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880908fa-2ea2-4581-afc9-5e2ecbc7a504 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:46.952248 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880908fa-2ea2-4581-afc9-5e2ecbc7a504 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:46.961854 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4696a-f614-451c-a22f-76fad727ed28 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:46.970436 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9bfc15-564b-4ff0-9f96-b8bbba598706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:46.971392 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bfc15-564b-4ff0-9f96-b8bbba598706 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:46.971681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bfc15-564b-4ff0-9f96-b8bbba598706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:46.972472 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9bfc15-564b-4ff0-9f96-b8bbba598706 admin admin] PATH_INFO: `/v3/projects/185bd59b265645adaaf38024e579825d/users/1c7f86e87e5940d08b8509c4db01663d/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:46.981771 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9bc2ae-8b4f-4be5-b4b0-af6ac8162c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:46.982681 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bc2ae-8b4f-4be5-b4b0-af6ac8162c3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:46.982963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bc2ae-8b4f-4be5-b4b0-af6ac8162c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:46.983238 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bc2ae-8b4f-4be5-b4b0-af6ac8162c3d admin admin] PATH_INFO: `/v3/projects/07d3e67c87d945b7aa3d135625239925/users/10e7373619ed483db3030760aa0e695e/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:46.985358 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9bc2ae-8b4f-4be5-b4b0-af6ac8162c3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=07d3e67c87d945b7aa3d135625239925, user_id=10e7373619ed483db3030760aa0e695e, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:46.988073 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9bc2ae-8b4f-4be5-b4b0-af6ac8162c3d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:47.008419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9bfc15-564b-4ff0-9f96-b8bbba598706 admin admin] RBAC: Authorizing `identity:create_grant(project_id=185bd59b265645adaaf38024e579825d, user_id=1c7f86e87e5940d08b8509c4db01663d, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:47.011831 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9bfc15-564b-4ff0-9f96-b8bbba598706 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:47.028239 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 900/1788] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:46 2022] PUT /identity/v3/projects/07d3e67c87d945b7aa3d135625239925/users/10e7373619ed483db3030760aa0e695e/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:47.030285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a545d-26dd-4e3f-b757-386744a731bf None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:47.030546 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a545d-26dd-4e3f-b757-386744a731bf None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:47.030832 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a545d-26dd-4e3f-b757-386744a731bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:47.061563 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:47.082130 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a36a545d-26dd-4e3f-b757-386744a731bf None None] MFA Rules not processed for user `10e7373619ed483db3030760aa0e695e`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:47.084649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a36a545d-26dd-4e3f-b757-386744a731bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:47.118126 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 889/1789] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:46 2022] PUT /identity/v3/projects/185bd59b265645adaaf38024e579825d/users/1c7f86e87e5940d08b8509c4db01663d/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:47.120390 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d3c41-35e5-4450-95e8-3fbe5761637f None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:47.120692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d3c41-35e5-4450-95e8-3fbe5761637f None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:47.121122 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d3c41-35e5-4450-95e8-3fbe5761637f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:47.206124 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-fc3d3c41-35e5-4450-95e8-3fbe5761637f None None] MFA Rules not processed for user `1c7f86e87e5940d08b8509c4db01663d`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:47.208587 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-fc3d3c41-35e5-4450-95e8-3fbe5761637f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:47.256935 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:47.330911 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 901/1790] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:47 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:47.338858 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a545d-26dd-4e3f-b757-386744a731bf None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:47.343255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a36a545d-26dd-4e3f-b757-386744a731bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:47.379436 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 890/1791] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:47 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:47.382965 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d3c41-35e5-4450-95e8-3fbe5761637f None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:47.458548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a545d-26dd-4e3f-b757-386744a731bf None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:47.550928 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e2d985-16c0-4577-b62e-958451c9d97f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:47.551681 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2d985-16c0-4577-b62e-958451c9d97f service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:47.551976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2d985-16c0-4577-b62e-958451c9d97f service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:47.552253 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e2d985-16c0-4577-b62e-958451c9d97f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:47.567300 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d3c41-35e5-4450-95e8-3fbe5761637f None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:47.596799 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3b9ec-55fb-410b-a53a-3278abc84b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:47.597503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3b9ec-55fb-410b-a53a-3278abc84b03 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:47.597808 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3b9ec-55fb-410b-a53a-3278abc84b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:47.598063 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3b9ec-55fb-410b-a53a-3278abc84b03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:47.598777 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3b9ec-55fb-410b-a53a-3278abc84b03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:47.606747 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3b9ec-55fb-410b-a53a-3278abc84b03 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:47.740374 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:47.810654 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 891/1792] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:33:47 2022] POST /identity/v3/projects => generated 355 bytes in 428 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:33:47.814334 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3b9ec-55fb-410b-a53a-3278abc84b03 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:47.834949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e2d985-16c0-4577-b62e-958451c9d97f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:47.836476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e2d985-16c0-4577-b62e-958451c9d97f service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:47.880107 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 902/1793] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:47 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:48.002979 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac1b09-288d-4bfc-b293-b1d85e1f355f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:48.003295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac1b09-288d-4bfc-b293-b1d85e1f355f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:48.003588 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edac1b09-288d-4bfc-b293-b1d85e1f355f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:48.024497 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3b9ec-55fb-410b-a53a-3278abc84b03 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:48.046492 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795c6f0-9f5d-4998-8150-e99dd2adfc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:48.047206 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795c6f0-9f5d-4998-8150-e99dd2adfc0c admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:48.049480 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795c6f0-9f5d-4998-8150-e99dd2adfc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:48.049779 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795c6f0-9f5d-4998-8150-e99dd2adfc0c admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:48.051707 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d795c6f0-9f5d-4998-8150-e99dd2adfc0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:48.060805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d795c6f0-9f5d-4998-8150-e99dd2adfc0c admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:48.102073 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-edac1b09-288d-4bfc-b293-b1d85e1f355f None None] MFA Rules not processed for user `eecc2db7b5754624b678e51dd9a37a46`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:48.119829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-edac1b09-288d-4bfc-b293-b1d85e1f355f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:48.262025 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 903/1794] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:33:48.267720 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac1b09-288d-4bfc-b293-b1d85e1f355f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:48.267852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 892/1795] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:33:47 2022] POST /identity/v3/users => generated 284 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:48.286209 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795c6f0-9f5d-4998-8150-e99dd2adfc0c admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:48.324257 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795c6f0-9f5d-4998-8150-e99dd2adfc0c admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:48.336556 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930a661-a94f-4f7f-a96f-27fd0dcff7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:48.338340 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930a661-a94f-4f7f-a96f-27fd0dcff7cb admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:48.339013 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930a661-a94f-4f7f-a96f-27fd0dcff7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:48.339287 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4930a661-a94f-4f7f-a96f-27fd0dcff7cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:48.344534 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4930a661-a94f-4f7f-a96f-27fd0dcff7cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:48.346832 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4930a661-a94f-4f7f-a96f-27fd0dcff7cb admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:48.376930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 893/1796] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:48.386582 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930a661-a94f-4f7f-a96f-27fd0dcff7cb admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:48.417795 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edac1b09-288d-4bfc-b293-b1d85e1f355f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:48.430072 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4930a661-a94f-4f7f-a96f-27fd0dcff7cb admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:48.432451 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194d4ab-1fd2-40e5-a4a5-233445143f48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:48.433459 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194d4ab-1fd2-40e5-a4a5-233445143f48 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:48.435123 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194d4ab-1fd2-40e5-a4a5-233445143f48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:48.435642 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8194d4ab-1fd2-40e5-a4a5-233445143f48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:48.441612 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559abbb-de42-4542-a981-f549262ba221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:48.442313 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8194d4ab-1fd2-40e5-a4a5-233445143f48 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:48.442432 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559abbb-de42-4542-a981-f549262ba221 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:48.442637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559abbb-de42-4542-a981-f549262ba221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:48.442927 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559abbb-de42-4542-a981-f549262ba221 admin admin] PATH_INFO: `/v3/projects/47338a23896140368354dee12beb93c4/users/78e7d3468d3f434e84bc32be32440b48/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:48.498115 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3559abbb-de42-4542-a981-f549262ba221 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47338a23896140368354dee12beb93c4, user_id=78e7d3468d3f434e84bc32be32440b48, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:48.510382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3559abbb-de42-4542-a981-f549262ba221 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:48.615931 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194d4ab-1fd2-40e5-a4a5-233445143f48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:48.619857 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8194d4ab-1fd2-40e5-a4a5-233445143f48 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:48.657414 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 904/1797] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:48 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:48.660157 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 894/1798] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:48 2022] PUT /identity/v3/projects/47338a23896140368354dee12beb93c4/users/78e7d3468d3f434e84bc32be32440b48/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:48.664604 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194d4ab-1fd2-40e5-a4a5-233445143f48 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:48.688415 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8194d4ab-1fd2-40e5-a4a5-233445143f48 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:48.800147 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559abbb-de42-4542-a981-f549262ba221 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:48.801960 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-3559abbb-de42-4542-a981-f549262ba221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:48.846188 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f931e376-c2f1-4a54-a943-f0068d948e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:48.847309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931e376-c2f1-4a54-a943-f0068d948e17 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:48.847866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931e376-c2f1-4a54-a943-f0068d948e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:48.848496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f931e376-c2f1-4a54-a943-f0068d948e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:48.849442 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f931e376-c2f1-4a54-a943-f0068d948e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:48.851431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f931e376-c2f1-4a54-a943-f0068d948e17 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:48.865004 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 905/1799] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:48.873941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f931e376-c2f1-4a54-a943-f0068d948e17 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:48.911701 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:48.947511 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f931e376-c2f1-4a54-a943-f0068d948e17 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:48.960386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51655412-8518-40ee-a32e-33fa1476833d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:48.961630 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51655412-8518-40ee-a32e-33fa1476833d admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:48.962745 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51655412-8518-40ee-a32e-33fa1476833d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:48.963224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51655412-8518-40ee-a32e-33fa1476833d admin admin] PATH_INFO: `/v3/projects/47338a23896140368354dee12beb93c4/users/78e7d3468d3f434e84bc32be32440b48/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:48.972779 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51655412-8518-40ee-a32e-33fa1476833d admin admin] RBAC: Authorizing `identity:create_grant(project_id=47338a23896140368354dee12beb93c4, user_id=78e7d3468d3f434e84bc32be32440b48, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:48.977624 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51655412-8518-40ee-a32e-33fa1476833d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:48.983358 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559abbb-de42-4542-a981-f549262ba221 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:49.010802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f0c8c-0745-4012-adb4-01175895c2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:49.011661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f0c8c-0745-4012-adb4-01175895c2bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:49.012233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f0c8c-0745-4012-adb4-01175895c2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:49.012519 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f0c8c-0745-4012-adb4-01175895c2bb admin admin] PATH_INFO: `/v3/projects/5a3fbfe074d847d49f31b80ac458defc` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:49.013421 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 906/1800] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:48 2022] PUT /identity/v3/projects/47338a23896140368354dee12beb93c4/users/78e7d3468d3f434e84bc32be32440b48/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:49.023866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ae3a8-216f-4005-926e-628ab99e9db6 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:49.024116 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f0c8c-0745-4012-adb4-01175895c2bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a3fbfe074d847d49f31b80ac458defc)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:49.024254 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ae3a8-216f-4005-926e-628ab99e9db6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:49.024541 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ae3a8-216f-4005-926e-628ab99e9db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:49.034207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f0c8c-0745-4012-adb4-01175895c2bb admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:49.114605 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-fa3ae3a8-216f-4005-926e-628ab99e9db6 None None] MFA Rules not processed for user `78e7d3468d3f434e84bc32be32440b48`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:49.120863 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fa3ae3a8-216f-4005-926e-628ab99e9db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:49.300582 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 907/1801] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:49 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:49.312446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9d5f0c8c-0745-4012-adb4-01175895c2bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a3fbfe074d847d49f31b80ac458defc', 'request_id': 'req-9d5f0c8c-0745-4012-adb4-01175895c2bb'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:49.314883 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ae3a8-216f-4005-926e-628ab99e9db6 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:49.317379 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fa3ae3a8-216f-4005-926e-628ab99e9db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:49.325985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9d5f0c8c-0745-4012-adb4-01175895c2bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a3fbfe074d847d49f31b80ac458defc', 'request_id': 'req-9d5f0c8c-0745-4012-adb4-01175895c2bb'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:49.341095 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9d5f0c8c-0745-4012-adb4-01175895c2bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a3fbfe074d847d49f31b80ac458defc', 'request_id': 'req-9d5f0c8c-0745-4012-adb4-01175895c2bb'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:49.400285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3ae3a8-216f-4005-926e-628ab99e9db6 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:49.460753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9d5f0c8c-0745-4012-adb4-01175895c2bb admin admin] The token cache is being invalidate because project 5a3fbfe074d847d49f31b80ac458defc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:33:49.461410 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9d5f0c8c-0745-4012-adb4-01175895c2bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:49.464183 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9d5f0c8c-0745-4012-adb4-01175895c2bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:49.467172 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 895/1802] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:33:48 2022] DELETE /identity/v3/projects/5a3fbfe074d847d49f31b80ac458defc => generated 0 bytes in 668 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:33:49.472159 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f0c8c-0745-4012-adb4-01175895c2bb admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:49.473775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-9d5f0c8c-0745-4012-adb4-01175895c2bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:49.481370 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dff23c-b9de-47ce-a92a-950acf6ca9e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:49.485446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dff23c-b9de-47ce-a92a-950acf6ca9e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:49.487338 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dff23c-b9de-47ce-a92a-950acf6ca9e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:49.487853 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dff23c-b9de-47ce-a92a-950acf6ca9e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:49.491018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-42dff23c-b9de-47ce-a92a-950acf6ca9e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:49.676414 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f0c8c-0745-4012-adb4-01175895c2bb admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:49.695841 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136beddc-c559-410f-b175-16f595043a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:49.697694 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136beddc-c559-410f-b175-16f595043a22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:49.698129 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136beddc-c559-410f-b175-16f595043a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:49.698548 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136beddc-c559-410f-b175-16f595043a22 admin admin] PATH_INFO: `/v3/users/3dcee11d98a24c3eaa24a3af3d5330cb` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:49.700173 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136beddc-c559-410f-b175-16f595043a22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dcee11d98a24c3eaa24a3af3d5330cb)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:49.702234 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136beddc-c559-410f-b175-16f595043a22 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:49.749734 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dff23c-b9de-47ce-a92a-950acf6ca9e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:49.751244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dff23c-b9de-47ce-a92a-950acf6ca9e6 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:49.784145 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 908/1803] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:49 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:33:50.016474 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-136beddc-c559-410f-b175-16f595043a22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dcee11d98a24c3eaa24a3af3d5330cb'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:50.100467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dff23c-b9de-47ce-a92a-950acf6ca9e6 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:50.102343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-42dff23c-b9de-47ce-a92a-950acf6ca9e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:50.112350 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-136beddc-c559-410f-b175-16f595043a22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dcee11d98a24c3eaa24a3af3d5330cb'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:50.136265 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dff23c-b9de-47ce-a92a-950acf6ca9e6 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:50.141396 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-136beddc-c559-410f-b175-16f595043a22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dcee11d98a24c3eaa24a3af3d5330cb'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:50.144800 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-136beddc-c559-410f-b175-16f595043a22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dcee11d98a24c3eaa24a3af3d5330cb'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:50.147089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-136beddc-c559-410f-b175-16f595043a22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dcee11d98a24c3eaa24a3af3d5330cb'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:50.151549 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b4d4ab-6726-482f-8543-1e7584561690 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:50.152401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4d4ab-6726-482f-8543-1e7584561690 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:50.154409 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4d4ab-6726-482f-8543-1e7584561690 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:50.158109 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b4d4ab-6726-482f-8543-1e7584561690 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:50.163736 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b8b4d4ab-6726-482f-8543-1e7584561690 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:50.196197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 896/1804] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:33:49 2022] DELETE /identity/v3/users/3dcee11d98a24c3eaa24a3af3d5330cb => generated 0 bytes in 724 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:33:50.423151 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b4d4ab-6726-482f-8543-1e7584561690 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:50.424713 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b4d4ab-6726-482f-8543-1e7584561690 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:50.450217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 909/1805] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:50 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:33:50.615763 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:50.967506 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136beddc-c559-410f-b175-16f595043a22 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:50.972631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-136beddc-c559-410f-b175-16f595043a22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:51.116075 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136beddc-c559-410f-b175-16f595043a22 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:51.125631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf024795-f426-4d41-aa48-d7fba56d6d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:51.126555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf024795-f426-4d41-aa48-d7fba56d6d52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:51.126834 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf024795-f426-4d41-aa48-d7fba56d6d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:51.127115 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf024795-f426-4d41-aa48-d7fba56d6d52 admin admin] PATH_INFO: `/v3/projects/e97500d3d01f4d908ac699c5347082cb` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:51.128476 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf024795-f426-4d41-aa48-d7fba56d6d52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e97500d3d01f4d908ac699c5347082cb)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:51.130501 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf024795-f426-4d41-aa48-d7fba56d6d52 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:51.259624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-cf024795-f426-4d41-aa48-d7fba56d6d52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e97500d3d01f4d908ac699c5347082cb', 'request_id': 'req-cf024795-f426-4d41-aa48-d7fba56d6d52'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:51.267312 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-cf024795-f426-4d41-aa48-d7fba56d6d52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e97500d3d01f4d908ac699c5347082cb', 'request_id': 'req-cf024795-f426-4d41-aa48-d7fba56d6d52'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:51.273432 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-cf024795-f426-4d41-aa48-d7fba56d6d52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e97500d3d01f4d908ac699c5347082cb', 'request_id': 'req-cf024795-f426-4d41-aa48-d7fba56d6d52'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:51.359336 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-cf024795-f426-4d41-aa48-d7fba56d6d52 admin admin] The token cache is being invalidate because project e97500d3d01f4d908ac699c5347082cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:33:51.359683 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-cf024795-f426-4d41-aa48-d7fba56d6d52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:51.360424 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-cf024795-f426-4d41-aa48-d7fba56d6d52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:51.362114 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 897/1806] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:33:50 2022] DELETE /identity/v3/projects/e97500d3d01f4d908ac699c5347082cb => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:51.366989 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a131d86-4ce5-46df-aa5c-56e353c48da3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:51.367511 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a131d86-4ce5-46df-aa5c-56e353c48da3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:51.367649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a131d86-4ce5-46df-aa5c-56e353c48da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:51.387174 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4a131d86-4ce5-46df-aa5c-56e353c48da3 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:51.388431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4a131d86-4ce5-46df-aa5c-56e353c48da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:51.452794 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 910/1807] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:51.462298 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61161702-dd96-4a8e-b3ab-45bbd01a1f00 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:51.465465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61161702-dd96-4a8e-b3ab-45bbd01a1f00 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:51.465760 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61161702-dd96-4a8e-b3ab-45bbd01a1f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:51.485628 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-61161702-dd96-4a8e-b3ab-45bbd01a1f00 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:51.487151 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-61161702-dd96-4a8e-b3ab-45bbd01a1f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:51.496494 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 898/1808] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:51.500765 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a131d86-4ce5-46df-aa5c-56e353c48da3 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:51.511117 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a131d86-4ce5-46df-aa5c-56e353c48da3 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:51.516899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8541e-669a-474d-bba1-48725c50c2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:51.517584 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8541e-669a-474d-bba1-48725c50c2f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:51.517877 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8541e-669a-474d-bba1-48725c50c2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:51.518154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b8541e-669a-474d-bba1-48725c50c2f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:51.518894 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8541e-669a-474d-bba1-48725c50c2f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:33:51.519205 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8541e-669a-474d-bba1-48725c50c2f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:51.521304 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b8541e-669a-474d-bba1-48725c50c2f5 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:51.535819 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 911/1809] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:33:51.540683 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61161702-dd96-4a8e-b3ab-45bbd01a1f00 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:51.550228 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61161702-dd96-4a8e-b3ab-45bbd01a1f00 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:51.555145 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3231394e-5f00-44a8-a8a0-4d4483fa84c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:51.556598 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231394e-5f00-44a8-a8a0-4d4483fa84c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:51.557145 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231394e-5f00-44a8-a8a0-4d4483fa84c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:51.557238 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3231394e-5f00-44a8-a8a0-4d4483fa84c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:51.557895 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3231394e-5f00-44a8-a8a0-4d4483fa84c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:51.559904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3231394e-5f00-44a8-a8a0-4d4483fa84c8 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:51.627727 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 899/1810] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:33:51 2022] POST /identity/v3/projects => generated 385 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:51.628832 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8541e-669a-474d-bba1-48725c50c2f5 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:51.639551 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b8541e-669a-474d-bba1-48725c50c2f5 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:51.704747 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c178a31-6abb-4e6e-8ca5-da891555b030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:51.705594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c178a31-6abb-4e6e-8ca5-da891555b030 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:51.706024 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c178a31-6abb-4e6e-8ca5-da891555b030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:51.706308 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c178a31-6abb-4e6e-8ca5-da891555b030 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:51.707288 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c178a31-6abb-4e6e-8ca5-da891555b030 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:51.709281 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c178a31-6abb-4e6e-8ca5-da891555b030 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:51.789607 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 912/1811] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:33:51 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:51.799578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3231394e-5f00-44a8-a8a0-4d4483fa84c8 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:51.813868 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3231394e-5f00-44a8-a8a0-4d4483fa84c8 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:51.820714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb83ec8-79eb-4b18-835f-f4e2344d971a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:51.821379 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb83ec8-79eb-4b18-835f-f4e2344d971a admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:51.822246 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb83ec8-79eb-4b18-835f-f4e2344d971a admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:51.823148 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb83ec8-79eb-4b18-835f-f4e2344d971a admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:51.824056 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb83ec8-79eb-4b18-835f-f4e2344d971a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:51.825666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb83ec8-79eb-4b18-835f-f4e2344d971a admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:51.837926 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 900/1812] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:51.841525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c178a31-6abb-4e6e-8ca5-da891555b030 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:51.852035 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c178a31-6abb-4e6e-8ca5-da891555b030 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:51.858305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc4909b-b0e5-48fa-9010-536ea6f6b73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:51.859026 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4909b-b0e5-48fa-9010-536ea6f6b73c admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:51.859226 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4909b-b0e5-48fa-9010-536ea6f6b73c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:51.859534 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc4909b-b0e5-48fa-9010-536ea6f6b73c admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:51.860265 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc4909b-b0e5-48fa-9010-536ea6f6b73c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:51.861876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc4909b-b0e5-48fa-9010-536ea6f6b73c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:51.878898 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 913/1813] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:33:51.880100 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb83ec8-79eb-4b18-835f-f4e2344d971a admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:51.890410 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb83ec8-79eb-4b18-835f-f4e2344d971a admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:51.895158 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c2cbc-1514-4768-abb1-f4c4102f112f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:51.896071 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c2cbc-1514-4768-abb1-f4c4102f112f admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:51.896357 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c2cbc-1514-4768-abb1-f4c4102f112f admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:51.896641 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c2cbc-1514-4768-abb1-f4c4102f112f admin admin] PATH_INFO: `/v3/projects/d27694c746b140339cd429412a96c5b9/users/8b756b6a20b649f68e8ec336ca4fc6e0/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:51.913665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433c2cbc-1514-4768-abb1-f4c4102f112f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d27694c746b140339cd429412a96c5b9, user_id=8b756b6a20b649f68e8ec336ca4fc6e0, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:51.917830 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433c2cbc-1514-4768-abb1-f4c4102f112f admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:52.012003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 901/1814] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:51 2022] PUT /identity/v3/projects/d27694c746b140339cd429412a96c5b9/users/8b756b6a20b649f68e8ec336ca4fc6e0/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:33:52.022501 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6764c6-2219-4e60-9465-38ef17b681aa None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:52.022824 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6764c6-2219-4e60-9465-38ef17b681aa None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:52.023103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6764c6-2219-4e60-9465-38ef17b681aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:52.027565 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:52.065168 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-aa6764c6-2219-4e60-9465-38ef17b681aa None None] MFA Rules not processed for user `8b756b6a20b649f68e8ec336ca4fc6e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:52.066483 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-aa6764c6-2219-4e60-9465-38ef17b681aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:52.192343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 914/1815] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:33:52.217764 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c2cbc-1514-4768-abb1-f4c4102f112f admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:52.220074 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-433c2cbc-1514-4768-abb1-f4c4102f112f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:52.289345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:52.298858 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c2cbc-1514-4768-abb1-f4c4102f112f admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:52.303244 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4dca2-5363-44c4-8899-5aa9433a6520 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:52.304392 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4dca2-5363-44c4-8899-5aa9433a6520 service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:52.304680 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4dca2-5363-44c4-8899-5aa9433a6520 service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:52.304990 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4dca2-5363-44c4-8899-5aa9433a6520 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:52.316092 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da4dca2-5363-44c4-8899-5aa9433a6520 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:52.317599 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da4dca2-5363-44c4-8899-5aa9433a6520 service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:52.343352 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 902/1816] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:52.355214 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6764c6-2219-4e60-9465-38ef17b681aa None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:52.435739 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6764c6-2219-4e60-9465-38ef17b681aa None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:52.445374 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814806c0-9935-4623-95f3-a2d52166aaf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:52.453040 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814806c0-9935-4623-95f3-a2d52166aaf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:52.453040 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814806c0-9935-4623-95f3-a2d52166aaf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:52.453040 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814806c0-9935-4623-95f3-a2d52166aaf2 admin admin] PATH_INFO: `/v3/users/8b756b6a20b649f68e8ec336ca4fc6e0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:52.453040 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814806c0-9935-4623-95f3-a2d52166aaf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b756b6a20b649f68e8ec336ca4fc6e0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:52.453040 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814806c0-9935-4623-95f3-a2d52166aaf2 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:52.482182 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:52.697547 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-814806c0-9935-4623-95f3-a2d52166aaf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b756b6a20b649f68e8ec336ca4fc6e0'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:52.698399 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-814806c0-9935-4623-95f3-a2d52166aaf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b756b6a20b649f68e8ec336ca4fc6e0'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:52.706237 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-814806c0-9935-4623-95f3-a2d52166aaf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b756b6a20b649f68e8ec336ca4fc6e0'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:52.707061 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-814806c0-9935-4623-95f3-a2d52166aaf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b756b6a20b649f68e8ec336ca4fc6e0'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:52.803400 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-814806c0-9935-4623-95f3-a2d52166aaf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b756b6a20b649f68e8ec336ca4fc6e0'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:52.828459 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 915/1817] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:33:52 2022] DELETE /identity/v3/users/8b756b6a20b649f68e8ec336ca4fc6e0 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:52.833196 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5828632-fbc3-49f3-bf13-9dc53d15a2b0 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:52.833489 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5828632-fbc3-49f3-bf13-9dc53d15a2b0 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:52.833776 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5828632-fbc3-49f3-bf13-9dc53d15a2b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:52.857439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-d5828632-fbc3-49f3-bf13-9dc53d15a2b0 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:52.859143 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d5828632-fbc3-49f3-bf13-9dc53d15a2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:52.940280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 903/1818] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:52.951066 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814806c0-9935-4623-95f3-a2d52166aaf2 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:52.952793 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-814806c0-9935-4623-95f3-a2d52166aaf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:53.015220 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814806c0-9935-4623-95f3-a2d52166aaf2 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:53.018776 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d649a432-a80d-49d0-a955-0669512c9400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:53.019464 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649a432-a80d-49d0-a955-0669512c9400 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:53.020120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649a432-a80d-49d0-a955-0669512c9400 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:53.020405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d649a432-a80d-49d0-a955-0669512c9400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:53.030389 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d649a432-a80d-49d0-a955-0669512c9400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:53.031872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d649a432-a80d-49d0-a955-0669512c9400 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:53.042436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 916/1819] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:53.102023 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:53.552175 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=83,keystone:INSERT=7,keystone:DELETE=10 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:53.600648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5828632-fbc3-49f3-bf13-9dc53d15a2b0 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:53.604769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d5828632-fbc3-49f3-bf13-9dc53d15a2b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:53.621621 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5828632-fbc3-49f3-bf13-9dc53d15a2b0 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:53.628315 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4184c16-b4fa-492d-99b8-77322a4579f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:53.629461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4184c16-b4fa-492d-99b8-77322a4579f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:53.629818 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4184c16-b4fa-492d-99b8-77322a4579f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:53.630264 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4184c16-b4fa-492d-99b8-77322a4579f3 admin admin] PATH_INFO: `/v3/projects/d27694c746b140339cd429412a96c5b9` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:53.633771 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4184c16-b4fa-492d-99b8-77322a4579f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d27694c746b140339cd429412a96c5b9)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:53.639236 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4184c16-b4fa-492d-99b8-77322a4579f3 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:53.671828 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:53.814767 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:53.896926 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e4184c16-b4fa-492d-99b8-77322a4579f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd27694c746b140339cd429412a96c5b9', 'request_id': 'req-e4184c16-b4fa-492d-99b8-77322a4579f3'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:53.913296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e4184c16-b4fa-492d-99b8-77322a4579f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd27694c746b140339cd429412a96c5b9', 'request_id': 'req-e4184c16-b4fa-492d-99b8-77322a4579f3'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:53.921908 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e4184c16-b4fa-492d-99b8-77322a4579f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd27694c746b140339cd429412a96c5b9', 'request_id': 'req-e4184c16-b4fa-492d-99b8-77322a4579f3'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:53.930663 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:53.957674 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:54.003721 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e4184c16-b4fa-492d-99b8-77322a4579f3 admin admin] The token cache is being invalidate because project d27694c746b140339cd429412a96c5b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:33:54.003865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e4184c16-b4fa-492d-99b8-77322a4579f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:54.004597 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e4184c16-b4fa-492d-99b8-77322a4579f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:33:54.006177 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 904/1820] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:33:53 2022] DELETE /identity/v3/projects/d27694c746b140339cd429412a96c5b9 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:33:54.011994 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410cfd5-a283-496c-bf05-e1c8d74f3174 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:54.012305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410cfd5-a283-496c-bf05-e1c8d74f3174 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:54.012635 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4410cfd5-a283-496c-bf05-e1c8d74f3174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:54.037029 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4410cfd5-a283-496c-bf05-e1c8d74f3174 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:54.038442 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4410cfd5-a283-496c-bf05-e1c8d74f3174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:54.109358 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 917/1821] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:33:54.118828 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd68a1-975d-4023-b91a-8005f8adc48e None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:54.119202 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd68a1-975d-4023-b91a-8005f8adc48e None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:54.119520 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd68a1-975d-4023-b91a-8005f8adc48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:54.137442 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:54.143446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-03bd68a1-975d-4023-b91a-8005f8adc48e None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:54.144801 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-03bd68a1-975d-4023-b91a-8005f8adc48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:54.153300 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 905/1822] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:54.157698 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:54.159906 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410cfd5-a283-496c-bf05-e1c8d74f3174 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:54.170181 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4410cfd5-a283-496c-bf05-e1c8d74f3174 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:54.175426 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785402eb-1dc8-4f41-820e-98326e21af41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:54.176098 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785402eb-1dc8-4f41-820e-98326e21af41 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:54.176382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785402eb-1dc8-4f41-820e-98326e21af41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:54.176657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785402eb-1dc8-4f41-820e-98326e21af41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:54.177387 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785402eb-1dc8-4f41-820e-98326e21af41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:33:54.177699 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785402eb-1dc8-4f41-820e-98326e21af41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:54.179368 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785402eb-1dc8-4f41-820e-98326e21af41 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:54.192075 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 918/1823] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:33:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:33:54.197319 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd68a1-975d-4023-b91a-8005f8adc48e None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:54.203312 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:54.206341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd68a1-975d-4023-b91a-8005f8adc48e None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:54.211109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109605dc-c2e7-45de-b819-2f8b141c4857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:54.211803 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109605dc-c2e7-45de-b819-2f8b141c4857 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:54.212156 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109605dc-c2e7-45de-b819-2f8b141c4857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:54.212436 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109605dc-c2e7-45de-b819-2f8b141c4857 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:54.213171 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109605dc-c2e7-45de-b819-2f8b141c4857 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:54.215081 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109605dc-c2e7-45de-b819-2f8b141c4857 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:54.226763 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:54.308688 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 906/1824] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:33:54 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:54.312876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785402eb-1dc8-4f41-820e-98326e21af41 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:54.323142 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785402eb-1dc8-4f41-820e-98326e21af41 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:54.381749 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc657403-cfc6-4c14-b96f-cddce9bdc6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:54.382454 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc657403-cfc6-4c14-b96f-cddce9bdc6b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:54.382738 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc657403-cfc6-4c14-b96f-cddce9bdc6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:54.383558 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc657403-cfc6-4c14-b96f-cddce9bdc6b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:54.384063 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc657403-cfc6-4c14-b96f-cddce9bdc6b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:54.385983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc657403-cfc6-4c14-b96f-cddce9bdc6b5 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:54.403881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:54.471863 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 919/1825] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:33:54 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:33:54.479076 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109605dc-c2e7-45de-b819-2f8b141c4857 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:54.489790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109605dc-c2e7-45de-b819-2f8b141c4857 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:54.495565 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5786ca-e68e-4b21-a83b-5b4bd4a1bdbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:54.496568 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5786ca-e68e-4b21-a83b-5b4bd4a1bdbf admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:54.496900 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5786ca-e68e-4b21-a83b-5b4bd4a1bdbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:54.497174 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5786ca-e68e-4b21-a83b-5b4bd4a1bdbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:54.498186 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5786ca-e68e-4b21-a83b-5b4bd4a1bdbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:54.499886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5786ca-e68e-4b21-a83b-5b4bd4a1bdbf admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:54.510973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 907/1826] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:33:54.515398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc657403-cfc6-4c14-b96f-cddce9bdc6b5 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:54.526119 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc657403-cfc6-4c14-b96f-cddce9bdc6b5 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:54.529908 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:54.532621 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dacad8-ed3b-4bfb-82fe-f8024e38af32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:54.533292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dacad8-ed3b-4bfb-82fe-f8024e38af32 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:54.533582 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dacad8-ed3b-4bfb-82fe-f8024e38af32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:54.533864 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dacad8-ed3b-4bfb-82fe-f8024e38af32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:54.534601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dacad8-ed3b-4bfb-82fe-f8024e38af32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:54.536249 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dacad8-ed3b-4bfb-82fe-f8024e38af32 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:54.547872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 920/1827] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:33:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:33:54.553667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:54.556790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5786ca-e68e-4b21-a83b-5b4bd4a1bdbf admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:54.566833 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5786ca-e68e-4b21-a83b-5b4bd4a1bdbf admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:54.572261 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd34e7-865e-472d-8a02-7da140ef446d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:54.574571 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd34e7-865e-472d-8a02-7da140ef446d admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:54.574571 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd34e7-865e-472d-8a02-7da140ef446d admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:54.574571 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcd34e7-865e-472d-8a02-7da140ef446d admin admin] PATH_INFO: `/v3/projects/f30204d8c4854913896a7249cef3f34e/users/6f06d3188a6f440798873553606e9d1b/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:54.590178 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd34e7-865e-472d-8a02-7da140ef446d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f30204d8c4854913896a7249cef3f34e, user_id=6f06d3188a6f440798873553606e9d1b, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:54.592883 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcd34e7-865e-472d-8a02-7da140ef446d admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:54.677902 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 908/1828] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:33:54 2022] PUT /identity/v3/projects/f30204d8c4854913896a7249cef3f34e/users/6f06d3188a6f440798873553606e9d1b/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:33:54.693367 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4cd02-c76a-462f-b0f6-e74864f6875e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:54.693832 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4cd02-c76a-462f-b0f6-e74864f6875e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:54.693942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec4cd02-c76a-462f-b0f6-e74864f6875e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:54.720606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1ec4cd02-c76a-462f-b0f6-e74864f6875e None None] MFA Rules not processed for user `6f06d3188a6f440798873553606e9d1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:54.721818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1ec4cd02-c76a-462f-b0f6-e74864f6875e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:54.785412 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 921/1829] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:33:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:54.796603 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd34e7-865e-472d-8a02-7da140ef446d admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:54.798385 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-7fcd34e7-865e-472d-8a02-7da140ef446d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:54.821143 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:54.855706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:54.859276 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcd34e7-865e-472d-8a02-7da140ef446d admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:54.862982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13eaa83-a0a1-4d1d-861e-d03bfb479978 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:54.863736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13eaa83-a0a1-4d1d-861e-d03bfb479978 service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:54.864023 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13eaa83-a0a1-4d1d-861e-d03bfb479978 service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:54.864298 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13eaa83-a0a1-4d1d-861e-d03bfb479978 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:54.872738 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13eaa83-a0a1-4d1d-861e-d03bfb479978 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:54.874158 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13eaa83-a0a1-4d1d-861e-d03bfb479978 service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:54.882843 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 909/1830] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:58.555922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7899290-a788-41d9-8e2c-2d8a9aec2537 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:58.555922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7899290-a788-41d9-8e2c-2d8a9aec2537 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:58.555922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7899290-a788-41d9-8e2c-2d8a9aec2537 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:58.555922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d7899290-a788-41d9-8e2c-2d8a9aec2537 None None] MFA Rules not processed for user `059b05007c0144f7a50dbd9f0ef41359`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:58.555922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d7899290-a788-41d9-8e2c-2d8a9aec2537 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:58.555922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 922/1831] 10.222.0.15 () {58 vars in 1015 bytes} [Thu Jun 30 10:33:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:33:58.557948 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13eaa83-a0a1-4d1d-861e-d03bfb479978 service cinder] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:58.557948 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-a13eaa83-a0a1-4d1d-861e-d03bfb479978 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:58.557948 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13eaa83-a0a1-4d1d-861e-d03bfb479978 service cinder] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:58.558570 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953949dc-494d-4f8b-b5e4-659934563576 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:58.558570 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953949dc-494d-4f8b-b5e4-659934563576 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:58.558570 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953949dc-494d-4f8b-b5e4-659934563576 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:58.558570 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953949dc-494d-4f8b-b5e4-659934563576 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:58.559566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953949dc-494d-4f8b-b5e4-659934563576 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:58.559566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953949dc-494d-4f8b-b5e4-659934563576 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:58.559566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 910/1832] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:33:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:33:58.568566 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d98544-8b30-4d45-b68b-cfd1937057b7 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:58.568968 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d98544-8b30-4d45-b68b-cfd1937057b7 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:58.569353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d98544-8b30-4d45-b68b-cfd1937057b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:58.607234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-90d98544-8b30-4d45-b68b-cfd1937057b7 None None] MFA Rules not processed for user `66ad8defb9b6453c96f1a8a7343695ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:33:58.607234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-90d98544-8b30-4d45-b68b-cfd1937057b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:58.645584 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:33:58.658916 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 923/1833] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jun 30 10:33:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:33:59.095964 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953949dc-494d-4f8b-b5e4-659934563576 service neutron] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:33:59.097582 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-953949dc-494d-4f8b-b5e4-659934563576 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:33:59.249869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953949dc-494d-4f8b-b5e4-659934563576 service neutron] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:33:59.258243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da99bad-4eab-4325-b6a2-a3fd0040b82a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07de926cb8594b098f79111dab8a9cc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:33:59.260521 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da99bad-4eab-4325-b6a2-a3fd0040b82a service swift] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:33:59.261332 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da99bad-4eab-4325-b6a2-a3fd0040b82a service swift] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:33:59.261812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da99bad-4eab-4325-b6a2-a3fd0040b82a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:33:59.276088 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da99bad-4eab-4325-b6a2-a3fd0040b82a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:33:59.278775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da99bad-4eab-4325-b6a2-a3fd0040b82a service swift] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:33:59.290890 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 911/1834] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jun 30 10:33:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:34:01.300754 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea10dc-4c8f-4176-9c58-307fc6c33fac None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:01.302286 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea10dc-4c8f-4176-9c58-307fc6c33fac None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:01.302581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eea10dc-4c8f-4176-9c58-307fc6c33fac None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:01.315416 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 924/1835] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jun 30 10:34:01 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:34:01.423246 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b2b1c-dcf4-484b-9d9e-94ed7302bfef None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:01.424189 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b2b1c-dcf4-484b-9d9e-94ed7302bfef None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:01.424622 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b2b1c-dcf4-484b-9d9e-94ed7302bfef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:01.449031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-181b2b1c-dcf4-484b-9d9e-94ed7302bfef None None] MFA Rules not processed for user `71f6d7715e6f49ffb9fdf6e1307bbd51`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:01.451663 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-181b2b1c-dcf4-484b-9d9e-94ed7302bfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:01.592760 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:01.640411 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 912/1836] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jun 30 10:34:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:02.022915 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea10dc-4c8f-4176-9c58-307fc6c33fac None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:02.022915 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8eea10dc-4c8f-4176-9c58-307fc6c33fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:02.085093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eea10dc-4c8f-4176-9c58-307fc6c33fac None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:02.104053 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8761f6d2-1b06-41ff-a838-24d0cef0f50a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:02.106888 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761f6d2-1b06-41ff-a838-24d0cef0f50a service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:02.107186 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761f6d2-1b06-41ff-a838-24d0cef0f50a service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:02.108946 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8761f6d2-1b06-41ff-a838-24d0cef0f50a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:02.149945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761f6d2-1b06-41ff-a838-24d0cef0f50a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:02.156456 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8761f6d2-1b06-41ff-a838-24d0cef0f50a service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:02.178114 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 925/1837] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:34:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:34:02.968390 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74dcf9-5dbd-4f02-a5bc-0417746aa826 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:02.968913 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74dcf9-5dbd-4f02-a5bc-0417746aa826 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:02.968990 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b74dcf9-5dbd-4f02-a5bc-0417746aa826 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:02.970933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 913/1838] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jun 30 10:34:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:34:02.984205 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde6d1e-922e-4b72-ab9e-748afe527efb None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:02.984925 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde6d1e-922e-4b72-ab9e-748afe527efb None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:02.985663 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde6d1e-922e-4b72-ab9e-748afe527efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:03.015722 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ebde6d1e-922e-4b72-ab9e-748afe527efb None None] MFA Rules not processed for user `71f6d7715e6f49ffb9fdf6e1307bbd51`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:03.017855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ebde6d1e-922e-4b72-ab9e-748afe527efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:03.034342 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 926/1839] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jun 30 10:34:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:03.050643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74dcf9-5dbd-4f02-a5bc-0417746aa826 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:03.052984 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-1b74dcf9-5dbd-4f02-a5bc-0417746aa826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:03.098064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b74dcf9-5dbd-4f02-a5bc-0417746aa826 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:03.118343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd869202-69ce-43aa-95af-bc6b5f9b0d5a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:03.119492 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd869202-69ce-43aa-95af-bc6b5f9b0d5a service nova] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:03.119787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd869202-69ce-43aa-95af-bc6b5f9b0d5a service nova] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:03.120122 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd869202-69ce-43aa-95af-bc6b5f9b0d5a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:03.140070 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd869202-69ce-43aa-95af-bc6b5f9b0d5a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:03.148831 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd869202-69ce-43aa-95af-bc6b5f9b0d5a service nova] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:03.158170 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 914/1840] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:34:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:34:13.011490 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=11 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:13.300040 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=14 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:25.967599 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde6d1e-922e-4b72-ab9e-748afe527efb None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:25.976188 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ebde6d1e-922e-4b72-ab9e-748afe527efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:26.066602 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde6d1e-922e-4b72-ab9e-748afe527efb None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:26.072339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a9b2fc-117a-421c-a771-0ebec0e30b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:26.073232 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9b2fc-117a-421c-a771-0ebec0e30b7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:26.073710 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9b2fc-117a-421c-a771-0ebec0e30b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:26.073988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a9b2fc-117a-421c-a771-0ebec0e30b7d admin admin] PATH_INFO: `/v3/users/b4da4c2e2cb743b89344c49b30deedc7` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:26.075518 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a9b2fc-117a-421c-a771-0ebec0e30b7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4da4c2e2cb743b89344c49b30deedc7)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:26.077743 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a9b2fc-117a-421c-a771-0ebec0e30b7d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:26.309591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-36a9b2fc-117a-421c-a771-0ebec0e30b7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4da4c2e2cb743b89344c49b30deedc7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:26.310269 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-36a9b2fc-117a-421c-a771-0ebec0e30b7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4da4c2e2cb743b89344c49b30deedc7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:26.318246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-36a9b2fc-117a-421c-a771-0ebec0e30b7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4da4c2e2cb743b89344c49b30deedc7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:26.319255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-36a9b2fc-117a-421c-a771-0ebec0e30b7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4da4c2e2cb743b89344c49b30deedc7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:26.408259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-36a9b2fc-117a-421c-a771-0ebec0e30b7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4da4c2e2cb743b89344c49b30deedc7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:26.426953 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 927/1841] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:34:25 2022] DELETE /identity/v3/users/b4da4c2e2cb743b89344c49b30deedc7 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:34:26.432221 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2ceb0-0e19-447e-b89a-dda1a135467e None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:26.432729 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2ceb0-0e19-447e-b89a-dda1a135467e None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:26.432947 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c2ceb0-0e19-447e-b89a-dda1a135467e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:26.453146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-95c2ceb0-0e19-447e-b89a-dda1a135467e None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:26.454576 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-95c2ceb0-0e19-447e-b89a-dda1a135467e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:26.522273 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 915/1842] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:26.532599 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a9b2fc-117a-421c-a771-0ebec0e30b7d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:26.534135 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-36a9b2fc-117a-421c-a771-0ebec0e30b7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:26.615178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a9b2fc-117a-421c-a771-0ebec0e30b7d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:26.619965 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4546f95-a439-49be-9861-b1e086e04aba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:26.620629 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4546f95-a439-49be-9861-b1e086e04aba service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:26.620938 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4546f95-a439-49be-9861-b1e086e04aba service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:26.621196 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4546f95-a439-49be-9861-b1e086e04aba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:26.630616 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4546f95-a439-49be-9861-b1e086e04aba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:26.632102 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4546f95-a439-49be-9861-b1e086e04aba service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:26.641870 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 928/1843] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:34:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:34:27.131994 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2ceb0-0e19-447e-b89a-dda1a135467e None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:27.133759 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-95c2ceb0-0e19-447e-b89a-dda1a135467e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:27.150970 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c2ceb0-0e19-447e-b89a-dda1a135467e None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:27.161366 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ff4244-4489-4b5d-8460-c271d0b11b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:27.163215 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff4244-4489-4b5d-8460-c271d0b11b98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:27.163701 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff4244-4489-4b5d-8460-c271d0b11b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:27.164255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ff4244-4489-4b5d-8460-c271d0b11b98 admin admin] PATH_INFO: `/v3/projects/3a5f238764b845f7bcb17cd15935b51b` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:27.166868 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff4244-4489-4b5d-8460-c271d0b11b98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a5f238764b845f7bcb17cd15935b51b)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:27.169363 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ff4244-4489-4b5d-8460-c271d0b11b98 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:27.329566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-72ff4244-4489-4b5d-8460-c271d0b11b98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a5f238764b845f7bcb17cd15935b51b', 'request_id': 'req-72ff4244-4489-4b5d-8460-c271d0b11b98'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:27.339978 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-72ff4244-4489-4b5d-8460-c271d0b11b98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a5f238764b845f7bcb17cd15935b51b', 'request_id': 'req-72ff4244-4489-4b5d-8460-c271d0b11b98'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:27.349921 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-72ff4244-4489-4b5d-8460-c271d0b11b98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a5f238764b845f7bcb17cd15935b51b', 'request_id': 'req-72ff4244-4489-4b5d-8460-c271d0b11b98'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:27.460385 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-72ff4244-4489-4b5d-8460-c271d0b11b98 admin admin] The token cache is being invalidate because project 3a5f238764b845f7bcb17cd15935b51b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:34:27.461018 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-72ff4244-4489-4b5d-8460-c271d0b11b98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:27.462855 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-72ff4244-4489-4b5d-8460-c271d0b11b98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:27.467819 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 916/1844] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:34:27 2022] DELETE /identity/v3/projects/3a5f238764b845f7bcb17cd15935b51b => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:27.469026 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4546f95-a439-49be-9861-b1e086e04aba service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:27.479840 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4546f95-a439-49be-9861-b1e086e04aba service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:27.511338 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:27.545716 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef1d17-4985-44e2-8546-ddd662d4090c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:27.549559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef1d17-4985-44e2-8546-ddd662d4090c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:27.549559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef1d17-4985-44e2-8546-ddd662d4090c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:27.549559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef1d17-4985-44e2-8546-ddd662d4090c admin admin] PATH_INFO: `/v3/users/28cc5bf620d94228bcb5359e7d8f9aab` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:27.549559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fef1d17-4985-44e2-8546-ddd662d4090c admin admin] RBAC: Authorizing `identity:delete_user(user_id=28cc5bf620d94228bcb5359e7d8f9aab)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:27.550353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fef1d17-4985-44e2-8546-ddd662d4090c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:27.578626 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4866643-9f7e-4386-b299-f69ef440781d None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:27.578933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4866643-9f7e-4386-b299-f69ef440781d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:27.579187 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4866643-9f7e-4386-b299-f69ef440781d None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:27.580725 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 917/1845] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jun 30 10:34:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:34:27.590339 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f804f-b042-419d-8cbd-d28ddaeac78f None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:27.591886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f804f-b042-419d-8cbd-d28ddaeac78f None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:27.592411 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3f804f-b042-419d-8cbd-d28ddaeac78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:27.613926 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-6b3f804f-b042-419d-8cbd-d28ddaeac78f None None] MFA Rules not processed for user `71f6d7715e6f49ffb9fdf6e1307bbd51`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:27.615255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-6b3f804f-b042-419d-8cbd-d28ddaeac78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:27.690016 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 918/1846] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jun 30 10:34:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:27.704068 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f804f-b042-419d-8cbd-d28ddaeac78f None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:27.706154 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-6b3f804f-b042-419d-8cbd-d28ddaeac78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:27.735038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3f804f-b042-419d-8cbd-d28ddaeac78f None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:27.745886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d61080-2613-4a6c-aaf4-7556f278ffc8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:27.748023 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d61080-2613-4a6c-aaf4-7556f278ffc8 service nova] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:27.748023 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d61080-2613-4a6c-aaf4-7556f278ffc8 service nova] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:27.748023 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d61080-2613-4a6c-aaf4-7556f278ffc8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:27.760701 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d61080-2613-4a6c-aaf4-7556f278ffc8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:27.762799 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d61080-2613-4a6c-aaf4-7556f278ffc8 service nova] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:27.777352 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 919/1847] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:34:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:34:27.790819 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8fef1d17-4985-44e2-8546-ddd662d4090c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28cc5bf620d94228bcb5359e7d8f9aab'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:27.791779 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8fef1d17-4985-44e2-8546-ddd662d4090c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28cc5bf620d94228bcb5359e7d8f9aab'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:27.799735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8fef1d17-4985-44e2-8546-ddd662d4090c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28cc5bf620d94228bcb5359e7d8f9aab'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:27.822985 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8fef1d17-4985-44e2-8546-ddd662d4090c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28cc5bf620d94228bcb5359e7d8f9aab'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:27.980013 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8fef1d17-4985-44e2-8546-ddd662d4090c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28cc5bf620d94228bcb5359e7d8f9aab'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:28.022124 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 929/1848] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:34:27 2022] DELETE /identity/v3/users/28cc5bf620d94228bcb5359e7d8f9aab => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:28.678933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d61080-2613-4a6c-aaf4-7556f278ffc8 service nova] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:28.680601 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-b6d61080-2613-4a6c-aaf4-7556f278ffc8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:28.689144 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:28.696234 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d61080-2613-4a6c-aaf4-7556f278ffc8 service nova] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:28.755610 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50b3f2-5c64-45d0-9726-b12c5afc9080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:28.756560 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50b3f2-5c64-45d0-9726-b12c5afc9080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:28.756873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50b3f2-5c64-45d0-9726-b12c5afc9080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:28.757155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50b3f2-5c64-45d0-9726-b12c5afc9080 admin admin] PATH_INFO: `/v3/projects/8421fba1161945f0964e23740fe9fa39` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:28.758977 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c50b3f2-5c64-45d0-9726-b12c5afc9080 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8421fba1161945f0964e23740fe9fa39)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:28.761048 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c50b3f2-5c64-45d0-9726-b12c5afc9080 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:28.879223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2c50b3f2-5c64-45d0-9726-b12c5afc9080 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8421fba1161945f0964e23740fe9fa39', 'request_id': 'req-2c50b3f2-5c64-45d0-9726-b12c5afc9080'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:28.887828 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2c50b3f2-5c64-45d0-9726-b12c5afc9080 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8421fba1161945f0964e23740fe9fa39', 'request_id': 'req-2c50b3f2-5c64-45d0-9726-b12c5afc9080'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:28.894049 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2c50b3f2-5c64-45d0-9726-b12c5afc9080 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8421fba1161945f0964e23740fe9fa39', 'request_id': 'req-2c50b3f2-5c64-45d0-9726-b12c5afc9080'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:29.005417 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2c50b3f2-5c64-45d0-9726-b12c5afc9080 admin admin] The token cache is being invalidate because project 8421fba1161945f0964e23740fe9fa39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:34:29.005720 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2c50b3f2-5c64-45d0-9726-b12c5afc9080 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:29.006416 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2c50b3f2-5c64-45d0-9726-b12c5afc9080 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:29.008955 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 920/1849] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:34:28 2022] DELETE /identity/v3/projects/8421fba1161945f0964e23740fe9fa39 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:29.013829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe5bd0-c0ca-4512-9b16-bf5203f1433f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:29.014323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe5bd0-c0ca-4512-9b16-bf5203f1433f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:29.014423 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe5bd0-c0ca-4512-9b16-bf5203f1433f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:29.035788 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a5fe5bd0-c0ca-4512-9b16-bf5203f1433f None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:29.036943 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a5fe5bd0-c0ca-4512-9b16-bf5203f1433f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:29.135582 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 930/1850] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:29.138230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bce7f-c32e-4675-855e-647d45c526e9 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:29.138230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bce7f-c32e-4675-855e-647d45c526e9 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:29.138230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2bce7f-c32e-4675-855e-647d45c526e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:29.160122 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-aa2bce7f-c32e-4675-855e-647d45c526e9 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:29.161503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-aa2bce7f-c32e-4675-855e-647d45c526e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:29.182430 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe5bd0-c0ca-4512-9b16-bf5203f1433f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:29.183021 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 921/1851] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:29.207641 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe5bd0-c0ca-4512-9b16-bf5203f1433f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:29.222033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7182c6-3c13-484f-bc72-75c5c251d11d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:29.222726 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7182c6-3c13-484f-bc72-75c5c251d11d admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:29.223015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7182c6-3c13-484f-bc72-75c5c251d11d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:29.224314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7182c6-3c13-484f-bc72-75c5c251d11d admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:29.224314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7182c6-3c13-484f-bc72-75c5c251d11d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:34:29.225598 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7182c6-3c13-484f-bc72-75c5c251d11d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:29.227524 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7182c6-3c13-484f-bc72-75c5c251d11d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:29.253127 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 931/1852] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:34:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:34:29.254056 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2bce7f-c32e-4675-855e-647d45c526e9 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:29.265530 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2bce7f-c32e-4675-855e-647d45c526e9 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:29.271167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4b010-38d0-4983-9276-8d3f6bde1914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:29.272139 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4b010-38d0-4983-9276-8d3f6bde1914 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:29.272478 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4b010-38d0-4983-9276-8d3f6bde1914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:29.272926 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e4b010-38d0-4983-9276-8d3f6bde1914 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:29.273640 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4b010-38d0-4983-9276-8d3f6bde1914 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:29.275865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e4b010-38d0-4983-9276-8d3f6bde1914 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:29.371650 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 922/1853] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:34:29 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:34:29.372829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7182c6-3c13-484f-bc72-75c5c251d11d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:29.384525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7182c6-3c13-484f-bc72-75c5c251d11d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:29.413188 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:29.447839 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861a354-cc9f-40cf-ad9a-1d628a1cac7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:29.448731 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861a354-cc9f-40cf-ad9a-1d628a1cac7f admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:29.449067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861a354-cc9f-40cf-ad9a-1d628a1cac7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:29.449356 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861a354-cc9f-40cf-ad9a-1d628a1cac7f admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:29.450059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861a354-cc9f-40cf-ad9a-1d628a1cac7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:29.452128 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861a354-cc9f-40cf-ad9a-1d628a1cac7f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:29.522718 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4b010-38d0-4983-9276-8d3f6bde1914 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:29.525232 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-87e4b010-38d0-4983-9276-8d3f6bde1914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:29.540327 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e4b010-38d0-4983-9276-8d3f6bde1914 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:29.547663 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725338dd-2f75-4882-a344-b58c346fd4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:29.548745 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725338dd-2f75-4882-a344-b58c346fd4bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:29.549300 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725338dd-2f75-4882-a344-b58c346fd4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:29.549727 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725338dd-2f75-4882-a344-b58c346fd4bc admin admin] PATH_INFO: `/v3/users/1c7f86e87e5940d08b8509c4db01663d` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:29.551213 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725338dd-2f75-4882-a344-b58c346fd4bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c7f86e87e5940d08b8509c4db01663d)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:29.554221 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725338dd-2f75-4882-a344-b58c346fd4bc admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:29.560094 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 932/1854] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:34:29 2022] POST /identity/v3/users => generated 294 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:34:29.564347 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861a354-cc9f-40cf-ad9a-1d628a1cac7f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:29.576278 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861a354-cc9f-40cf-ad9a-1d628a1cac7f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:29.581580 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20690288-3c58-4807-8681-cc65139a4177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:29.582540 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20690288-3c58-4807-8681-cc65139a4177 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:29.582834 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20690288-3c58-4807-8681-cc65139a4177 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:29.591595 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20690288-3c58-4807-8681-cc65139a4177 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:29.591595 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20690288-3c58-4807-8681-cc65139a4177 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:29.591595 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20690288-3c58-4807-8681-cc65139a4177 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:29.597806 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 933/1855] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:34:29.602495 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20690288-3c58-4807-8681-cc65139a4177 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:29.613786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20690288-3c58-4807-8681-cc65139a4177 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:29.625547 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578ecd4-3ac2-4d88-9720-b541620ee7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:29.626566 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578ecd4-3ac2-4d88-9720-b541620ee7f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:29.631672 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578ecd4-3ac2-4d88-9720-b541620ee7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:29.631672 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578ecd4-3ac2-4d88-9720-b541620ee7f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:29.631672 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578ecd4-3ac2-4d88-9720-b541620ee7f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:29.631672 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578ecd4-3ac2-4d88-9720-b541620ee7f7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:29.640523 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 934/1856] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:34:29.644660 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578ecd4-3ac2-4d88-9720-b541620ee7f7 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:29.655150 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578ecd4-3ac2-4d88-9720-b541620ee7f7 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:29.659877 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49dd0cd-f1b0-482d-b5b8-35b06d23862b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:29.660773 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49dd0cd-f1b0-482d-b5b8-35b06d23862b admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:29.661059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49dd0cd-f1b0-482d-b5b8-35b06d23862b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:29.661338 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49dd0cd-f1b0-482d-b5b8-35b06d23862b admin admin] PATH_INFO: `/v3/projects/a0166e46cee64c97a3cd718b81117aa2/users/c5dca6ad91f6480997650b133bda9581/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:29.676886 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49dd0cd-f1b0-482d-b5b8-35b06d23862b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0166e46cee64c97a3cd718b81117aa2, user_id=c5dca6ad91f6480997650b133bda9581, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:29.679884 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49dd0cd-f1b0-482d-b5b8-35b06d23862b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:29.776156 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-725338dd-2f75-4882-a344-b58c346fd4bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c7f86e87e5940d08b8509c4db01663d'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:29.792448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 935/1857] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:34:29 2022] PUT /identity/v3/projects/a0166e46cee64c97a3cd718b81117aa2/users/c5dca6ad91f6480997650b133bda9581/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:34:29.802555 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac235b4-a7f8-4e72-86bf-d854774fe811 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:29.803132 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac235b4-a7f8-4e72-86bf-d854774fe811 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:29.803440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac235b4-a7f8-4e72-86bf-d854774fe811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:29.830705 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-aac235b4-a7f8-4e72-86bf-d854774fe811 None None] MFA Rules not processed for user `c5dca6ad91f6480997650b133bda9581`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:29.832007 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-aac235b4-a7f8-4e72-86bf-d854774fe811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:29.878341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-725338dd-2f75-4882-a344-b58c346fd4bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c7f86e87e5940d08b8509c4db01663d'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:29.886804 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:29.896104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-725338dd-2f75-4882-a344-b58c346fd4bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c7f86e87e5940d08b8509c4db01663d'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:29.897261 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-725338dd-2f75-4882-a344-b58c346fd4bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c7f86e87e5940d08b8509c4db01663d'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:29.898301 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-725338dd-2f75-4882-a344-b58c346fd4bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c7f86e87e5940d08b8509c4db01663d'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:29.918990 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 923/1858] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:34:29 2022] DELETE /identity/v3/users/1c7f86e87e5940d08b8509c4db01663d => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:34:29.925069 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbab56a-a602-4885-9fb0-93309c741cf9 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:29.925511 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbab56a-a602-4885-9fb0-93309c741cf9 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:29.925997 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbab56a-a602-4885-9fb0-93309c741cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:29.939321 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 936/1859] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:34:29.945092 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac235b4-a7f8-4e72-86bf-d854774fe811 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:29.947455 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-2dbab56a-a602-4885-9fb0-93309c741cf9 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:29.948682 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-2dbab56a-a602-4885-9fb0-93309c741cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:29.952509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-aac235b4-a7f8-4e72-86bf-d854774fe811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:30.054804 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac235b4-a7f8-4e72-86bf-d854774fe811 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:30.056095 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 924/1860] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:30.063839 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531fd32-27a4-43f8-b282-92b5f4dee4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:30.064378 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531fd32-27a4-43f8-b282-92b5f4dee4be admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:30.064665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531fd32-27a4-43f8-b282-92b5f4dee4be admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:30.064950 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531fd32-27a4-43f8-b282-92b5f4dee4be admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:30.065976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531fd32-27a4-43f8-b282-92b5f4dee4be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:30.067111 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbab56a-a602-4885-9fb0-93309c741cf9 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:30.068042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531fd32-27a4-43f8-b282-92b5f4dee4be admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:30.075628 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-2dbab56a-a602-4885-9fb0-93309c741cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:30.161354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 937/1861] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:34:29 2022] POST /identity/v3/projects => generated 389 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:34:30.165647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531fd32-27a4-43f8-b282-92b5f4dee4be admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:30.175413 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbab56a-a602-4885-9fb0-93309c741cf9 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:30.240619 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31928b-5108-4c03-8355-392e422edb38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:30.241451 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31928b-5108-4c03-8355-392e422edb38 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:30.241742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31928b-5108-4c03-8355-392e422edb38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:30.242021 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a31928b-5108-4c03-8355-392e422edb38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:30.291052 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531fd32-27a4-43f8-b282-92b5f4dee4be admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:30.320251 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26128ad-18f9-4358-aa61-d841b6b47932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:30.321460 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26128ad-18f9-4358-aa61-d841b6b47932 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:30.321726 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26128ad-18f9-4358-aa61-d841b6b47932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:30.322289 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26128ad-18f9-4358-aa61-d841b6b47932 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:30.322952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26128ad-18f9-4358-aa61-d841b6b47932 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:30.325105 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26128ad-18f9-4358-aa61-d841b6b47932 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:30.382016 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31928b-5108-4c03-8355-392e422edb38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:30.384804 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a31928b-5108-4c03-8355-392e422edb38 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:30.386086 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:30.418022 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 925/1862] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:34:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:34:30.439243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 938/1863] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:34:30 2022] POST /identity/v3/users => generated 301 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:34:30.443408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31928b-5108-4c03-8355-392e422edb38 service neutron] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:30.461442 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a31928b-5108-4c03-8355-392e422edb38 service neutron] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:30.471696 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8199c66-481c-47f4-9b60-5c52fc6036b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:30.472559 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8199c66-481c-47f4-9b60-5c52fc6036b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:30.472996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8199c66-481c-47f4-9b60-5c52fc6036b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:30.473287 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8199c66-481c-47f4-9b60-5c52fc6036b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:30.479656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8199c66-481c-47f4-9b60-5c52fc6036b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:30.479656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8199c66-481c-47f4-9b60-5c52fc6036b6 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:30.507249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 926/1864] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:34:30.512546 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26128ad-18f9-4358-aa61-d841b6b47932 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:30.524717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26128ad-18f9-4358-aa61-d841b6b47932 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:30.531066 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c24ead-e816-4fee-8703-71513cb063ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:30.532139 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c24ead-e816-4fee-8703-71513cb063ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:30.532493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c24ead-e816-4fee-8703-71513cb063ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:30.532971 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c24ead-e816-4fee-8703-71513cb063ae admin admin] PATH_INFO: `/v3/projects/388983cd535243d2b67ab1efb4e2dd15/users/23bfbe045f834025b7d74248a0cc686a/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:30.561276 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c24ead-e816-4fee-8703-71513cb063ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=388983cd535243d2b67ab1efb4e2dd15, user_id=23bfbe045f834025b7d74248a0cc686a, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:30.564132 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c24ead-e816-4fee-8703-71513cb063ae admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:30.706195 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 939/1865] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:34:30 2022] PUT /identity/v3/projects/388983cd535243d2b67ab1efb4e2dd15/users/23bfbe045f834025b7d74248a0cc686a/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:30.710121 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8199c66-481c-47f4-9b60-5c52fc6036b6 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:30.720255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8199c66-481c-47f4-9b60-5c52fc6036b6 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:30.780006 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b4a1d-52e6-4154-a2b2-9cf0b0ca3a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:30.780787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b4a1d-52e6-4154-a2b2-9cf0b0ca3a1a admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:30.780947 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b4a1d-52e6-4154-a2b2-9cf0b0ca3a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:30.781293 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b4a1d-52e6-4154-a2b2-9cf0b0ca3a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:30.781973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b4a1d-52e6-4154-a2b2-9cf0b0ca3a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:30.783741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04b4a1d-52e6-4154-a2b2-9cf0b0ca3a1a admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:30.794600 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 927/1866] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:30 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:34:30.799586 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c24ead-e816-4fee-8703-71513cb063ae admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:30.812905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c24ead-e816-4fee-8703-71513cb063ae admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:30.819179 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca96987-ea09-404b-b132-c08f3e0569fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:30.820385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca96987-ea09-404b-b132-c08f3e0569fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:30.821627 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca96987-ea09-404b-b132-c08f3e0569fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:30.822208 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca96987-ea09-404b-b132-c08f3e0569fb admin admin] PATH_INFO: `/v3/projects/388983cd535243d2b67ab1efb4e2dd15/users/23bfbe045f834025b7d74248a0cc686a/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:30.824480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca96987-ea09-404b-b132-c08f3e0569fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=388983cd535243d2b67ab1efb4e2dd15, user_id=23bfbe045f834025b7d74248a0cc686a, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:30.827556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca96987-ea09-404b-b132-c08f3e0569fb admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:30.839510 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 940/1867] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:34:30 2022] PUT /identity/v3/projects/388983cd535243d2b67ab1efb4e2dd15/users/23bfbe045f834025b7d74248a0cc686a/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:30.850613 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fe4a0-e7ff-4685-9f69-57cc68117033 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:30.851101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fe4a0-e7ff-4685-9f69-57cc68117033 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:30.851608 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1fe4a0-e7ff-4685-9f69-57cc68117033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:30.891567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-ff1fe4a0-e7ff-4685-9f69-57cc68117033 None None] MFA Rules not processed for user `23bfbe045f834025b7d74248a0cc686a`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:30.893110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-ff1fe4a0-e7ff-4685-9f69-57cc68117033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:30.907087 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:DELETE=2,keystone:SELECT=98 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:30.966879 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 928/1868] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:30 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:30.978807 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca96987-ea09-404b-b132-c08f3e0569fb admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:30.980672 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3ca96987-ea09-404b-b132-c08f3e0569fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:31.013541 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fe4a0-e7ff-4685-9f69-57cc68117033 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:31.015481 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-ff1fe4a0-e7ff-4685-9f69-57cc68117033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:31.068427 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca96987-ea09-404b-b132-c08f3e0569fb admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:31.073563 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473184f5-29fa-4505-91dd-e391651465c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:31.074230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473184f5-29fa-4505-91dd-e391651465c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:31.074514 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473184f5-29fa-4505-91dd-e391651465c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:31.074795 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473184f5-29fa-4505-91dd-e391651465c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:31.100086 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473184f5-29fa-4505-91dd-e391651465c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:31.103467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473184f5-29fa-4505-91dd-e391651465c9 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:31.127560 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 941/1869] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:34:30 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:34:31.128612 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1fe4a0-e7ff-4685-9f69-57cc68117033 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:31.135574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8610d-fd6e-4ce3-93d8-61b450d030cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:31.137119 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8610d-fd6e-4ce3-93d8-61b450d030cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:31.137525 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8610d-fd6e-4ce3-93d8-61b450d030cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:31.137863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8610d-fd6e-4ce3-93d8-61b450d030cd admin admin] PATH_INFO: `/v3/projects/185bd59b265645adaaf38024e579825d` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:31.140093 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a8610d-fd6e-4ce3-93d8-61b450d030cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=185bd59b265645adaaf38024e579825d)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:31.148125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a8610d-fd6e-4ce3-93d8-61b450d030cd admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:31.274697 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473184f5-29fa-4505-91dd-e391651465c9 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:31.285536 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473184f5-29fa-4505-91dd-e391651465c9 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:31.291257 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d44645-82b6-4825-b582-3cfff2150869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:31.292096 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d44645-82b6-4825-b582-3cfff2150869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:31.292378 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d44645-82b6-4825-b582-3cfff2150869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:31.292651 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d44645-82b6-4825-b582-3cfff2150869 admin admin] PATH_INFO: `/v3/users/c5dca6ad91f6480997650b133bda9581` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:31.298835 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d44645-82b6-4825-b582-3cfff2150869 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5dca6ad91f6480997650b133bda9581)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:31.300836 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d44645-82b6-4825-b582-3cfff2150869 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:31.336934 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-c0a8610d-fd6e-4ce3-93d8-61b450d030cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '185bd59b265645adaaf38024e579825d', 'request_id': 'req-c0a8610d-fd6e-4ce3-93d8-61b450d030cd'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:31.343376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-c0a8610d-fd6e-4ce3-93d8-61b450d030cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '185bd59b265645adaaf38024e579825d', 'request_id': 'req-c0a8610d-fd6e-4ce3-93d8-61b450d030cd'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:31.348930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-c0a8610d-fd6e-4ce3-93d8-61b450d030cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '185bd59b265645adaaf38024e579825d', 'request_id': 'req-c0a8610d-fd6e-4ce3-93d8-61b450d030cd'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:31.432907 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-c0a8610d-fd6e-4ce3-93d8-61b450d030cd admin admin] The token cache is being invalidate because project 185bd59b265645adaaf38024e579825d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:34:31.433200 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-c0a8610d-fd6e-4ce3-93d8-61b450d030cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:31.433863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-c0a8610d-fd6e-4ce3-93d8-61b450d030cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:31.435630 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 929/1870] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:34:31 2022] DELETE /identity/v3/projects/185bd59b265645adaaf38024e579825d => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:31.439496 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8610d-fd6e-4ce3-93d8-61b450d030cd admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:31.441625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-c0a8610d-fd6e-4ce3-93d8-61b450d030cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:31.443175 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:31.507226 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:INSERT=6,keystone:DELETE=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:31.511766 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8610d-fd6e-4ce3-93d8-61b450d030cd admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:31.517109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4946ffb-ccc1-4f23-a14c-a60b27676624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:31.518035 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e4d44645-82b6-4825-b582-3cfff2150869 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5dca6ad91f6480997650b133bda9581'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:31.518219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4946ffb-ccc1-4f23-a14c-a60b27676624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:31.518631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4946ffb-ccc1-4f23-a14c-a60b27676624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:31.518743 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e4d44645-82b6-4825-b582-3cfff2150869 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5dca6ad91f6480997650b133bda9581'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:31.519393 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4946ffb-ccc1-4f23-a14c-a60b27676624 admin admin] PATH_INFO: `/v3/users/78e7d3468d3f434e84bc32be32440b48` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:31.520828 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4946ffb-ccc1-4f23-a14c-a60b27676624 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78e7d3468d3f434e84bc32be32440b48)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:31.523636 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4946ffb-ccc1-4f23-a14c-a60b27676624 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:31.526877 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e4d44645-82b6-4825-b582-3cfff2150869 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5dca6ad91f6480997650b133bda9581'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:31.527580 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e4d44645-82b6-4825-b582-3cfff2150869 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5dca6ad91f6480997650b133bda9581'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:31.630922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e4d44645-82b6-4825-b582-3cfff2150869 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5dca6ad91f6480997650b133bda9581'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:31.651739 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 942/1871] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:34:31 2022] DELETE /identity/v3/users/c5dca6ad91f6480997650b133bda9581 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:34:31.652591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d44645-82b6-4825-b582-3cfff2150869 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:31.654086 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e4d44645-82b6-4825-b582-3cfff2150869 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:31.748344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d44645-82b6-4825-b582-3cfff2150869 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:31.749369 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a5de6-1c81-41e4-b7a0-090b57d9d914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:31.750254 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a5de6-1c81-41e4-b7a0-090b57d9d914 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:31.751573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a5de6-1c81-41e4-b7a0-090b57d9d914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:31.751573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a5de6-1c81-41e4-b7a0-090b57d9d914 admin admin] PATH_INFO: `/v3/users/6f06d3188a6f440798873553606e9d1b` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:31.752149 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42a5de6-1c81-41e4-b7a0-090b57d9d914 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f06d3188a6f440798873553606e9d1b)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:31.754158 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42a5de6-1c81-41e4-b7a0-090b57d9d914 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:31.782967 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a4946ffb-ccc1-4f23-a14c-a60b27676624 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78e7d3468d3f434e84bc32be32440b48'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:31.871074 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a4946ffb-ccc1-4f23-a14c-a60b27676624 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78e7d3468d3f434e84bc32be32440b48'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:31.884113 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a4946ffb-ccc1-4f23-a14c-a60b27676624 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78e7d3468d3f434e84bc32be32440b48'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:31.884813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a4946ffb-ccc1-4f23-a14c-a60b27676624 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78e7d3468d3f434e84bc32be32440b48'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:31.885461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a4946ffb-ccc1-4f23-a14c-a60b27676624 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78e7d3468d3f434e84bc32be32440b48'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:31.898729 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 930/1872] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:34:31 2022] DELETE /identity/v3/users/78e7d3468d3f434e84bc32be32440b48 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:34:31.900303 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faedb9c-0694-4ed3-acfe-6a8daddc464b None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:31.900602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faedb9c-0694-4ed3-acfe-6a8daddc464b None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:31.900878 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faedb9c-0694-4ed3-acfe-6a8daddc464b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:31.906949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:31.920250 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-6faedb9c-0694-4ed3-acfe-6a8daddc464b None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:31.921637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-6faedb9c-0694-4ed3-acfe-6a8daddc464b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:31.997253 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 931/1873] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:32.001406 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a42a5de6-1c81-41e4-b7a0-090b57d9d914 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f06d3188a6f440798873553606e9d1b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:32.002234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a42a5de6-1c81-41e4-b7a0-090b57d9d914 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f06d3188a6f440798873553606e9d1b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:32.010300 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a42a5de6-1c81-41e4-b7a0-090b57d9d914 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f06d3188a6f440798873553606e9d1b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:32.011053 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a42a5de6-1c81-41e4-b7a0-090b57d9d914 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f06d3188a6f440798873553606e9d1b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:32.014593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faedb9c-0694-4ed3-acfe-6a8daddc464b None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:32.017388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-6faedb9c-0694-4ed3-acfe-6a8daddc464b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:32.079046 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faedb9c-0694-4ed3-acfe-6a8daddc464b None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:32.082890 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74d9306-6780-4a42-8b4b-a4c74d5c21b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:32.083594 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d9306-6780-4a42-8b4b-a4c74d5c21b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:32.083887 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d9306-6780-4a42-8b4b-a4c74d5c21b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:32.084168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d9306-6780-4a42-8b4b-a4c74d5c21b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:32.085967 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-f74d9306-6780-4a42-8b4b-a4c74d5c21b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:32.098423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d9306-6780-4a42-8b4b-a4c74d5c21b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:32.099893 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74d9306-6780-4a42-8b4b-a4c74d5c21b5 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:32.110195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 932/1874] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:34:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:34:32.122134 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a42a5de6-1c81-41e4-b7a0-090b57d9d914 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f06d3188a6f440798873553606e9d1b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:32.141524 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 943/1875] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:34:31 2022] DELETE /identity/v3/users/6f06d3188a6f440798873553606e9d1b => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:32.146130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02202f52-4142-480b-9528-f439f4e2e4d9 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:32.146424 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02202f52-4142-480b-9528-f439f4e2e4d9 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:32.147358 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02202f52-4142-480b-9528-f439f4e2e4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:32.166310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-02202f52-4142-480b-9528-f439f4e2e4d9 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:32.167544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-02202f52-4142-480b-9528-f439f4e2e4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:32.228736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 933/1876] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:32.238393 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a5de6-1c81-41e4-b7a0-090b57d9d914 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:32.239888 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a42a5de6-1c81-41e4-b7a0-090b57d9d914 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:32.300911 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:32.303406 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a5de6-1c81-41e4-b7a0-090b57d9d914 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:32.307100 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956dbea-98be-477d-864f-8d8b0fb8f54f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:32.308062 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956dbea-98be-477d-864f-8d8b0fb8f54f service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:32.308345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956dbea-98be-477d-864f-8d8b0fb8f54f service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:32.308708 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956dbea-98be-477d-864f-8d8b0fb8f54f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:32.318089 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3956dbea-98be-477d-864f-8d8b0fb8f54f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:32.319560 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3956dbea-98be-477d-864f-8d8b0fb8f54f service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:32.335492 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 944/1877] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:34:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:34:32.551492 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02202f52-4142-480b-9528-f439f4e2e4d9 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:32.552917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-02202f52-4142-480b-9528-f439f4e2e4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:32.565085 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02202f52-4142-480b-9528-f439f4e2e4d9 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:32.575738 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe786a98-6711-4a9a-a5fb-8f6dd29a4f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:32.576650 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe786a98-6711-4a9a-a5fb-8f6dd29a4f6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:32.576938 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe786a98-6711-4a9a-a5fb-8f6dd29a4f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:32.577212 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe786a98-6711-4a9a-a5fb-8f6dd29a4f6c admin admin] PATH_INFO: `/v3/projects/47338a23896140368354dee12beb93c4` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:32.578558 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe786a98-6711-4a9a-a5fb-8f6dd29a4f6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=47338a23896140368354dee12beb93c4)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:32.580457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe786a98-6711-4a9a-a5fb-8f6dd29a4f6c admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:32.638370 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956dbea-98be-477d-864f-8d8b0fb8f54f service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:32.640044 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3956dbea-98be-477d-864f-8d8b0fb8f54f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:32.662078 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956dbea-98be-477d-864f-8d8b0fb8f54f service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:32.667549 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc16199-2a1c-4288-8ba1-e451d5a97ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:32.668500 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc16199-2a1c-4288-8ba1-e451d5a97ff5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:32.668855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc16199-2a1c-4288-8ba1-e451d5a97ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:32.669303 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc16199-2a1c-4288-8ba1-e451d5a97ff5 admin admin] PATH_INFO: `/v3/projects/a0166e46cee64c97a3cd718b81117aa2` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:32.670895 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc16199-2a1c-4288-8ba1-e451d5a97ff5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0166e46cee64c97a3cd718b81117aa2)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:32.673035 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc16199-2a1c-4288-8ba1-e451d5a97ff5 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:32.712725 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:32.722352 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-fe786a98-6711-4a9a-a5fb-8f6dd29a4f6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47338a23896140368354dee12beb93c4', 'request_id': 'req-fe786a98-6711-4a9a-a5fb-8f6dd29a4f6c'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:32.729787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-fe786a98-6711-4a9a-a5fb-8f6dd29a4f6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47338a23896140368354dee12beb93c4', 'request_id': 'req-fe786a98-6711-4a9a-a5fb-8f6dd29a4f6c'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:32.735700 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-fe786a98-6711-4a9a-a5fb-8f6dd29a4f6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47338a23896140368354dee12beb93c4', 'request_id': 'req-fe786a98-6711-4a9a-a5fb-8f6dd29a4f6c'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:32.793805 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-acc16199-2a1c-4288-8ba1-e451d5a97ff5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0166e46cee64c97a3cd718b81117aa2', 'request_id': 'req-acc16199-2a1c-4288-8ba1-e451d5a97ff5'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:32.809270 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-fe786a98-6711-4a9a-a5fb-8f6dd29a4f6c admin admin] The token cache is being invalidate because project 47338a23896140368354dee12beb93c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:34:32.811392 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-fe786a98-6711-4a9a-a5fb-8f6dd29a4f6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:32.812164 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-fe786a98-6711-4a9a-a5fb-8f6dd29a4f6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:32.813879 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 934/1878] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:34:32 2022] DELETE /identity/v3/projects/47338a23896140368354dee12beb93c4 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:32.818825 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4425d-e85f-475f-b702-740363bb3aba None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:32.819117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4425d-e85f-475f-b702-740363bb3aba None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:32.819436 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c4425d-e85f-475f-b702-740363bb3aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:32.842384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-36c4425d-e85f-475f-b702-740363bb3aba None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:32.843605 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-36c4425d-e85f-475f-b702-740363bb3aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:32.920896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-acc16199-2a1c-4288-8ba1-e451d5a97ff5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0166e46cee64c97a3cd718b81117aa2', 'request_id': 'req-acc16199-2a1c-4288-8ba1-e451d5a97ff5'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:32.941783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 935/1879] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:34:32.941970 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-acc16199-2a1c-4288-8ba1-e451d5a97ff5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0166e46cee64c97a3cd718b81117aa2', 'request_id': 'req-acc16199-2a1c-4288-8ba1-e451d5a97ff5'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:32.942080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c4425d-e85f-475f-b702-740363bb3aba None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:32.942080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-36c4425d-e85f-475f-b702-740363bb3aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:32.963757 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c4425d-e85f-475f-b702-740363bb3aba None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:32.971479 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980d2ae-14d8-4877-a253-e48883da3f55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:32.972575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980d2ae-14d8-4877-a253-e48883da3f55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:32.972866 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980d2ae-14d8-4877-a253-e48883da3f55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:32.973164 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980d2ae-14d8-4877-a253-e48883da3f55 admin admin] PATH_INFO: `/v3/projects/f30204d8c4854913896a7249cef3f34e` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:32.974556 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980d2ae-14d8-4877-a253-e48883da3f55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f30204d8c4854913896a7249cef3f34e)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:32.977010 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980d2ae-14d8-4877-a253-e48883da3f55 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:32.987904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-acc16199-2a1c-4288-8ba1-e451d5a97ff5 admin admin] The token cache is being invalidate because project a0166e46cee64c97a3cd718b81117aa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:34:32.988202 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-acc16199-2a1c-4288-8ba1-e451d5a97ff5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:32.989218 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-acc16199-2a1c-4288-8ba1-e451d5a97ff5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:32.991381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 945/1880] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:34:32 2022] DELETE /identity/v3/projects/a0166e46cee64c97a3cd718b81117aa2 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:32.994880 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4332e-7ee4-417f-ad9b-2a6d22dcaa23 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:32.995395 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4332e-7ee4-417f-ad9b-2a6d22dcaa23 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:32.995814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce4332e-7ee4-417f-ad9b-2a6d22dcaa23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:33.018207 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-2ce4332e-7ee4-417f-ad9b-2a6d22dcaa23 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:33.019625 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2ce4332e-7ee4-417f-ad9b-2a6d22dcaa23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:33.028904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 946/1881] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:33.030111 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce4332e-7ee4-417f-ad9b-2a6d22dcaa23 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:33.034116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2ce4332e-7ee4-417f-ad9b-2a6d22dcaa23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:33.048536 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce4332e-7ee4-417f-ad9b-2a6d22dcaa23 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:33.054766 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787896d2-ff07-42f2-a765-68f938e801cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:33.056146 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787896d2-ff07-42f2-a765-68f938e801cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:33.056429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787896d2-ff07-42f2-a765-68f938e801cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:33.056715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787896d2-ff07-42f2-a765-68f938e801cc admin admin] PATH_INFO: `/v3/users/23bfbe045f834025b7d74248a0cc686a` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:33.058003 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787896d2-ff07-42f2-a765-68f938e801cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=23bfbe045f834025b7d74248a0cc686a)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:33.060079 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787896d2-ff07-42f2-a765-68f938e801cc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:33.155966 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1980d2ae-14d8-4877-a253-e48883da3f55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f30204d8c4854913896a7249cef3f34e', 'request_id': 'req-1980d2ae-14d8-4877-a253-e48883da3f55'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:33.162255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1980d2ae-14d8-4877-a253-e48883da3f55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f30204d8c4854913896a7249cef3f34e', 'request_id': 'req-1980d2ae-14d8-4877-a253-e48883da3f55'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:33.168438 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1980d2ae-14d8-4877-a253-e48883da3f55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f30204d8c4854913896a7249cef3f34e', 'request_id': 'req-1980d2ae-14d8-4877-a253-e48883da3f55'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:33.236641 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1980d2ae-14d8-4877-a253-e48883da3f55 admin admin] The token cache is being invalidate because project f30204d8c4854913896a7249cef3f34e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:34:33.236927 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1980d2ae-14d8-4877-a253-e48883da3f55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:33.243781 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1980d2ae-14d8-4877-a253-e48883da3f55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:33.245532 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 936/1882] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:34:32 2022] DELETE /identity/v3/projects/f30204d8c4854913896a7249cef3f34e => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:33.246477 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980d2ae-14d8-4877-a253-e48883da3f55 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:33.247887 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-1980d2ae-14d8-4877-a253-e48883da3f55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:33.281422 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-787896d2-ff07-42f2-a765-68f938e801cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23bfbe045f834025b7d74248a0cc686a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:33.282166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-787896d2-ff07-42f2-a765-68f938e801cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23bfbe045f834025b7d74248a0cc686a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:33.289253 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-787896d2-ff07-42f2-a765-68f938e801cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23bfbe045f834025b7d74248a0cc686a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:33.290345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-787896d2-ff07-42f2-a765-68f938e801cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23bfbe045f834025b7d74248a0cc686a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:33.302269 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:33.315128 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980d2ae-14d8-4877-a253-e48883da3f55 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:33.320393 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478f057-ddfd-446e-a1c2-08100e9a61a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:33.321721 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478f057-ddfd-446e-a1c2-08100e9a61a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:33.322003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478f057-ddfd-446e-a1c2-08100e9a61a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:33.322279 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478f057-ddfd-446e-a1c2-08100e9a61a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:33.322993 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478f057-ddfd-446e-a1c2-08100e9a61a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:34:33.323360 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478f057-ddfd-446e-a1c2-08100e9a61a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:33.324950 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f478f057-ddfd-446e-a1c2-08100e9a61a1 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:33.338616 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 937/1883] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:34:33.339901 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c191eb-819b-41da-a9aa-24fe397408f6 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:33.340192 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c191eb-819b-41da-a9aa-24fe397408f6 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:33.340474 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c191eb-819b-41da-a9aa-24fe397408f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:33.358368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-28c191eb-819b-41da-a9aa-24fe397408f6 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:33.359521 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-28c191eb-819b-41da-a9aa-24fe397408f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:33.367724 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 938/1884] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:33.368383 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c191eb-819b-41da-a9aa-24fe397408f6 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:33.369574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-28c191eb-819b-41da-a9aa-24fe397408f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:33.374549 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-787896d2-ff07-42f2-a765-68f938e801cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23bfbe045f834025b7d74248a0cc686a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:33.381170 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c191eb-819b-41da-a9aa-24fe397408f6 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:33.386691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199c346-b183-41f0-b1d6-cdb1a01dbb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:33.387389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199c346-b183-41f0-b1d6-cdb1a01dbb7a admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:33.387689 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199c346-b183-41f0-b1d6-cdb1a01dbb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:33.387959 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199c346-b183-41f0-b1d6-cdb1a01dbb7a admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:33.388657 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d199c346-b183-41f0-b1d6-cdb1a01dbb7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:33.390599 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d199c346-b183-41f0-b1d6-cdb1a01dbb7a admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:33.394171 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 947/1885] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:34:33 2022] DELETE /identity/v3/users/23bfbe045f834025b7d74248a0cc686a => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:33.398359 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4116e6-4226-4dc0-a4dd-06072f091bf2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:33.398672 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4116e6-4226-4dc0-a4dd-06072f091bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:33.399509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4116e6-4226-4dc0-a4dd-06072f091bf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:33.418912 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-fe4116e6-4226-4dc0-a4dd-06072f091bf2 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:33.420173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fe4116e6-4226-4dc0-a4dd-06072f091bf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:33.450232 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:33.496923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 948/1886] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:33.503014 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 939/1887] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:34:33 2022] POST /identity/v3/projects => generated 369 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:34:33.505608 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199c346-b183-41f0-b1d6-cdb1a01dbb7a admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:33.507583 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4116e6-4226-4dc0-a4dd-06072f091bf2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:33.518800 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4116e6-4226-4dc0-a4dd-06072f091bf2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:33.579771 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd6ef6-a0ec-4f31-96ea-e9caec5b01db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:33.580438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd6ef6-a0ec-4f31-96ea-e9caec5b01db admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:33.580560 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199c346-b183-41f0-b1d6-cdb1a01dbb7a admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:33.580722 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd6ef6-a0ec-4f31-96ea-e9caec5b01db admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:33.581013 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd6ef6-a0ec-4f31-96ea-e9caec5b01db admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:33.581770 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcd6ef6-a0ec-4f31-96ea-e9caec5b01db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:34:33.582085 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcd6ef6-a0ec-4f31-96ea-e9caec5b01db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:33.583764 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcd6ef6-a0ec-4f31-96ea-e9caec5b01db admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:33.585282 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9e1e3-5cb8-40d6-bc11-fa321814b921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:33.585960 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9e1e3-5cb8-40d6-bc11-fa321814b921 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:33.586875 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9e1e3-5cb8-40d6-bc11-fa321814b921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:33.587251 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb9e1e3-5cb8-40d6-bc11-fa321814b921 admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:33.587974 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb9e1e3-5cb8-40d6-bc11-fa321814b921 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:33.589876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb9e1e3-5cb8-40d6-bc11-fa321814b921 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:33.596459 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 949/1888] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:34:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:34:33.600838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd6ef6-a0ec-4f31-96ea-e9caec5b01db admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:33.611910 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd6ef6-a0ec-4f31-96ea-e9caec5b01db admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:33.616718 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e644eb2-5dfb-46bc-9733-06131c476dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:33.617916 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e644eb2-5dfb-46bc-9733-06131c476dcf admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:33.618356 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e644eb2-5dfb-46bc-9733-06131c476dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:33.618773 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e644eb2-5dfb-46bc-9733-06131c476dcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:33.619539 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e644eb2-5dfb-46bc-9733-06131c476dcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:33.621458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e644eb2-5dfb-46bc-9733-06131c476dcf admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:33.696517 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 940/1889] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:34:33 2022] POST /identity/v3/users => generated 285 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:34:33.697624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9e1e3-5cb8-40d6-bc11-fa321814b921 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:33.709484 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb9e1e3-5cb8-40d6-bc11-fa321814b921 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:33.723374 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 950/1890] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:34:33 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:34:33.725393 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e644eb2-5dfb-46bc-9733-06131c476dcf admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:33.804676 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e644eb2-5dfb-46bc-9733-06131c476dcf admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:33.805066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc48f9-cd01-42c8-82dc-c9f6caa10752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:33.807301 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc48f9-cd01-42c8-82dc-c9f6caa10752 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:33.807690 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc48f9-cd01-42c8-82dc-c9f6caa10752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:33.808029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edc48f9-cd01-42c8-82dc-c9f6caa10752 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:33.808879 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc48f9-cd01-42c8-82dc-c9f6caa10752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:33.811213 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edc48f9-cd01-42c8-82dc-c9f6caa10752 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:33.816790 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13499a8f-79f9-4d0e-acc0-200c63344516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:33.816977 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13499a8f-79f9-4d0e-acc0-200c63344516 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:33.817222 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13499a8f-79f9-4d0e-acc0-200c63344516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:33.817497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13499a8f-79f9-4d0e-acc0-200c63344516 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:33.818202 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13499a8f-79f9-4d0e-acc0-200c63344516 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:33.820563 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13499a8f-79f9-4d0e-acc0-200c63344516 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:33.825785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 941/1891] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:34:33.826895 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc48f9-cd01-42c8-82dc-c9f6caa10752 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:33.838854 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edc48f9-cd01-42c8-82dc-c9f6caa10752 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:33.843752 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1cb25-5ba6-41ff-9bf3-e8b81c7aa628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:33.844423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1cb25-5ba6-41ff-9bf3-e8b81c7aa628 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:33.844706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1cb25-5ba6-41ff-9bf3-e8b81c7aa628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:33.844983 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1cb25-5ba6-41ff-9bf3-e8b81c7aa628 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:33.845711 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1cb25-5ba6-41ff-9bf3-e8b81c7aa628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:33.847322 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1cb25-5ba6-41ff-9bf3-e8b81c7aa628 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:33.860445 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 942/1892] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:34:33.862755 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1cb25-5ba6-41ff-9bf3-e8b81c7aa628 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:33.873013 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1cb25-5ba6-41ff-9bf3-e8b81c7aa628 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:33.877806 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3526d2a-4721-454b-bc63-8b4fda457fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:33.878680 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3526d2a-4721-454b-bc63-8b4fda457fcd admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:33.878966 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3526d2a-4721-454b-bc63-8b4fda457fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:33.879297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3526d2a-4721-454b-bc63-8b4fda457fcd admin admin] PATH_INFO: `/v3/projects/d9d9b6a7fb7846a0874e86f7e20da02e/users/4453d4ca6665403292f6cbfbf57d9cae/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:33.893829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3526d2a-4721-454b-bc63-8b4fda457fcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9d9b6a7fb7846a0874e86f7e20da02e, user_id=4453d4ca6665403292f6cbfbf57d9cae, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:33.896533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3526d2a-4721-454b-bc63-8b4fda457fcd admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:33.969906 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 951/1893] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:34:33 2022] POST /identity/v3/users => generated 292 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:34:33.970575 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13499a8f-79f9-4d0e-acc0-200c63344516 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:33.972174 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-13499a8f-79f9-4d0e-acc0-200c63344516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:33.985041 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13499a8f-79f9-4d0e-acc0-200c63344516 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:33.989792 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0b721-16f8-4a2d-9198-617dd1a3f40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:33.990647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0b721-16f8-4a2d-9198-617dd1a3f40e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:33.990929 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0b721-16f8-4a2d-9198-617dd1a3f40e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:33.991204 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0b721-16f8-4a2d-9198-617dd1a3f40e admin admin] PATH_INFO: `/v3/projects/388983cd535243d2b67ab1efb4e2dd15` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:33.992506 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff0b721-16f8-4a2d-9198-617dd1a3f40e admin admin] RBAC: Authorizing `identity:delete_project(project_id=388983cd535243d2b67ab1efb4e2dd15)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:33.994521 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff0b721-16f8-4a2d-9198-617dd1a3f40e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:34.009659 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 943/1894] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:34:33 2022] PUT /identity/v3/projects/d9d9b6a7fb7846a0874e86f7e20da02e/users/4453d4ca6665403292f6cbfbf57d9cae/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:34:34.010815 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3526d2a-4721-454b-bc63-8b4fda457fcd admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:34.026617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:34.089789 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:34.095596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3526d2a-4721-454b-bc63-8b4fda457fcd admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:34.100371 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21adc4-d879-41c2-bfd2-9ab08457d499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:34.101057 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21adc4-d879-41c2-bfd2-9ab08457d499 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:34.101346 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21adc4-d879-41c2-bfd2-9ab08457d499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:34.101627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21adc4-d879-41c2-bfd2-9ab08457d499 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:34.102403 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21adc4-d879-41c2-bfd2-9ab08457d499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:34.104096 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21adc4-d879-41c2-bfd2-9ab08457d499 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:34.117692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 944/1895] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:34:34.119245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0920e-b13b-439e-af94-9941fd8c0ad7 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:34.119663 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0920e-b13b-439e-af94-9941fd8c0ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:34.120331 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0920e-b13b-439e-af94-9941fd8c0ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:34.142108 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8ff0b721-16f8-4a2d-9198-617dd1a3f40e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '388983cd535243d2b67ab1efb4e2dd15', 'request_id': 'req-8ff0b721-16f8-4a2d-9198-617dd1a3f40e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:34.163832 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-66d0920e-b13b-439e-af94-9941fd8c0ad7 None None] MFA Rules not processed for user `4453d4ca6665403292f6cbfbf57d9cae`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:34.165109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-66d0920e-b13b-439e-af94-9941fd8c0ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:34.230986 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8ff0b721-16f8-4a2d-9198-617dd1a3f40e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '388983cd535243d2b67ab1efb4e2dd15', 'request_id': 'req-8ff0b721-16f8-4a2d-9198-617dd1a3f40e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:34.238213 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8ff0b721-16f8-4a2d-9198-617dd1a3f40e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '388983cd535243d2b67ab1efb4e2dd15', 'request_id': 'req-8ff0b721-16f8-4a2d-9198-617dd1a3f40e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:34.248104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 945/1896] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:34 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:34.248601 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0920e-b13b-439e-af94-9941fd8c0ad7 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:34.261051 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8ff0b721-16f8-4a2d-9198-617dd1a3f40e admin admin] The token cache is being invalidate because project 388983cd535243d2b67ab1efb4e2dd15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:34:34.261360 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8ff0b721-16f8-4a2d-9198-617dd1a3f40e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:34.262141 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8ff0b721-16f8-4a2d-9198-617dd1a3f40e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:34.263981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 952/1897] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:34:33 2022] DELETE /identity/v3/projects/388983cd535243d2b67ab1efb4e2dd15 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:34.264739 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0b721-16f8-4a2d-9198-617dd1a3f40e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:34.266198 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8ff0b721-16f8-4a2d-9198-617dd1a3f40e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:34.369281 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0b721-16f8-4a2d-9198-617dd1a3f40e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:34.371697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d0920e-b13b-439e-af94-9941fd8c0ad7 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:34.377434 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c835b8e-9c93-4138-b8a2-9b285fe51172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:34.378052 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6873f4f-5035-4a8b-951c-50610390bb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:34.378745 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6873f4f-5035-4a8b-951c-50610390bb1c admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:34.379259 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6873f4f-5035-4a8b-951c-50610390bb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:34.379664 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6873f4f-5035-4a8b-951c-50610390bb1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:34.380412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6873f4f-5035-4a8b-951c-50610390bb1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:34.382262 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6873f4f-5035-4a8b-951c-50610390bb1c admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:34.391112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c835b8e-9c93-4138-b8a2-9b285fe51172 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:34.391560 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c835b8e-9c93-4138-b8a2-9b285fe51172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:34.391867 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c835b8e-9c93-4138-b8a2-9b285fe51172 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:34.393028 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c835b8e-9c93-4138-b8a2-9b285fe51172 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:34.395013 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c835b8e-9c93-4138-b8a2-9b285fe51172 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:34.403000 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 946/1898] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:34:34.405607 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef83ffd-3ddf-4d4b-856a-6b6250c91369 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:34.406939 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef83ffd-3ddf-4d4b-856a-6b6250c91369 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:34.407528 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef83ffd-3ddf-4d4b-856a-6b6250c91369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:34.434586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-eef83ffd-3ddf-4d4b-856a-6b6250c91369 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:34.436256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-eef83ffd-3ddf-4d4b-856a-6b6250c91369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:34.444813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 947/1899] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:34.446288 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef83ffd-3ddf-4d4b-856a-6b6250c91369 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:34.450303 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-eef83ffd-3ddf-4d4b-856a-6b6250c91369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:34.462420 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef83ffd-3ddf-4d4b-856a-6b6250c91369 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:34.480087 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d54a1-4373-46db-8a3c-4ec3f5220ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:34.481066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d54a1-4373-46db-8a3c-4ec3f5220ec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:34.481507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d54a1-4373-46db-8a3c-4ec3f5220ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:34.482086 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499d54a1-4373-46db-8a3c-4ec3f5220ec0 admin admin] PATH_INFO: `/v3/projects/78f728f0f9af407183ba0b45d9689d0f/users/3571d5ce8fbb4421b8af34b8a43d6fe4/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:34.511237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d54a1-4373-46db-8a3c-4ec3f5220ec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78f728f0f9af407183ba0b45d9689d0f, user_id=3571d5ce8fbb4421b8af34b8a43d6fe4, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:34.514907 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499d54a1-4373-46db-8a3c-4ec3f5220ec0 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:34.517157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 953/1900] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:34:34 2022] POST /identity/v3/projects => generated 369 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:34:34.519879 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836e5b2-2bf8-4963-bfee-eafaad75338a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:34.519879 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836e5b2-2bf8-4963-bfee-eafaad75338a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:34.519879 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b836e5b2-2bf8-4963-bfee-eafaad75338a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:34.538522 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b836e5b2-2bf8-4963-bfee-eafaad75338a None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:34.539898 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b836e5b2-2bf8-4963-bfee-eafaad75338a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:34.585799 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 948/1901] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:34:34 2022] PUT /identity/v3/projects/78f728f0f9af407183ba0b45d9689d0f/users/3571d5ce8fbb4421b8af34b8a43d6fe4/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:34.586837 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d54a1-4373-46db-8a3c-4ec3f5220ec0 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:34.592588 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:34.630898 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 954/1902] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:34.632571 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d0c5e-2c85-42bf-ac62-7ce3b7a49de2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:34.633758 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d0c5e-2c85-42bf-ac62-7ce3b7a49de2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:34.634283 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0d0c5e-2c85-42bf-ac62-7ce3b7a49de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:34.674668 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0e0d0c5e-2c85-42bf-ac62-7ce3b7a49de2 None None] MFA Rules not processed for user `3571d5ce8fbb4421b8af34b8a43d6fe4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:34.675948 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0e0d0c5e-2c85-42bf-ac62-7ce3b7a49de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:34.687759 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499d54a1-4373-46db-8a3c-4ec3f5220ec0 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:34.699374 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561318a-0f07-4869-862f-2f52933173cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:34.701318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561318a-0f07-4869-862f-2f52933173cc admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:34.701746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561318a-0f07-4869-862f-2f52933173cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:34.702091 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4561318a-0f07-4869-862f-2f52933173cc admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:34.703040 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561318a-0f07-4869-862f-2f52933173cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:34.706384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4561318a-0f07-4869-862f-2f52933173cc admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:34.729030 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:34.780465 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 955/1903] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:34 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:34.780465 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d0c5e-2c85-42bf-ac62-7ce3b7a49de2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:34.798323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 949/1904] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:34:34 2022] POST /identity/v3/users => generated 291 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:34:34.799947 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561318a-0f07-4869-862f-2f52933173cc admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:34.807952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0d0c5e-2c85-42bf-ac62-7ce3b7a49de2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:34.813485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4561318a-0f07-4869-862f-2f52933173cc admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:34.814008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b52f7-9202-4956-a191-9f2c8089f6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:34.814836 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b52f7-9202-4956-a191-9f2c8089f6fc admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:34.815297 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b52f7-9202-4956-a191-9f2c8089f6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:34.815641 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538b52f7-9202-4956-a191-9f2c8089f6fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:34.816948 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b52f7-9202-4956-a191-9f2c8089f6fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:34:34.816948 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b52f7-9202-4956-a191-9f2c8089f6fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:34.818819 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538b52f7-9202-4956-a191-9f2c8089f6fc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:34.819215 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e6d43-4bf2-4670-a6be-726e0e4e25db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:34.819936 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e6d43-4bf2-4670-a6be-726e0e4e25db admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:34.820228 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e6d43-4bf2-4670-a6be-726e0e4e25db admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:34.820593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1e6d43-4bf2-4670-a6be-726e0e4e25db admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:34.822656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e6d43-4bf2-4670-a6be-726e0e4e25db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:34.825167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1e6d43-4bf2-4670-a6be-726e0e4e25db admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:34.836557 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 956/1905] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:34:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:34:34.838065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b52f7-9202-4956-a191-9f2c8089f6fc admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:34.852504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538b52f7-9202-4956-a191-9f2c8089f6fc admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:34.857640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c433484-ea17-4d5c-855d-f367cb703f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:34.858393 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c433484-ea17-4d5c-855d-f367cb703f09 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:34.858756 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c433484-ea17-4d5c-855d-f367cb703f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:34.859108 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c433484-ea17-4d5c-855d-f367cb703f09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:34.859955 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c433484-ea17-4d5c-855d-f367cb703f09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:34.861635 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c433484-ea17-4d5c-855d-f367cb703f09 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:34.874030 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 957/1906] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:34:34.875506 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c433484-ea17-4d5c-855d-f367cb703f09 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:34.918401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c433484-ea17-4d5c-855d-f367cb703f09 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:34.918401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0634f86f-688f-4a60-a9a6-b00b39e22f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:34.918401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634f86f-688f-4a60-a9a6-b00b39e22f45 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:34.918401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634f86f-688f-4a60-a9a6-b00b39e22f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:34.919209 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634f86f-688f-4a60-a9a6-b00b39e22f45 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:34.919209 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0634f86f-688f-4a60-a9a6-b00b39e22f45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:34.919209 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0634f86f-688f-4a60-a9a6-b00b39e22f45 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:34.973714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 950/1907] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:34:34 2022] POST /identity/v3/projects => generated 385 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:34:34.973714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e6d43-4bf2-4670-a6be-726e0e4e25db admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:35.002855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 958/1908] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:34:34 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:34:35.002855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0634f86f-688f-4a60-a9a6-b00b39e22f45 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:35.069422 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1e6d43-4bf2-4670-a6be-726e0e4e25db admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:35.094378 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0634f86f-688f-4a60-a9a6-b00b39e22f45 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:35.099470 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec8f60-0fca-4248-8245-755e1e8d76f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:35.100149 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec8f60-0fca-4248-8245-755e1e8d76f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:35.100436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec8f60-0fca-4248-8245-755e1e8d76f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:35.100840 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec8f60-0fca-4248-8245-755e1e8d76f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:35.101565 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec8f60-0fca-4248-8245-755e1e8d76f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:35.104147 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec8f60-0fca-4248-8245-755e1e8d76f4 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:35.131728 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e57e43-d555-4bda-87eb-afd91711502d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:35.132688 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e57e43-d555-4bda-87eb-afd91711502d admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:35.133012 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e57e43-d555-4bda-87eb-afd91711502d admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:35.133297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e57e43-d555-4bda-87eb-afd91711502d admin admin] PATH_INFO: `/v3/projects/48068f2b2fc243f08ae1688ed32d7bc1/users/a8bb21e782d7442e928c995fad79f2d1/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:35.151532 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e57e43-d555-4bda-87eb-afd91711502d admin admin] RBAC: Authorizing `identity:create_grant(project_id=48068f2b2fc243f08ae1688ed32d7bc1, user_id=a8bb21e782d7442e928c995fad79f2d1, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:35.154243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e57e43-d555-4bda-87eb-afd91711502d admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:35.200832 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 959/1909] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:34:34 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:34:35.201671 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec8f60-0fca-4248-8245-755e1e8d76f4 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:35.221226 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec8f60-0fca-4248-8245-755e1e8d76f4 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:35.227468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f52945-8519-4188-9e67-a89687a072ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:35.229397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f52945-8519-4188-9e67-a89687a072ae admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:35.229690 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f52945-8519-4188-9e67-a89687a072ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:35.229977 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f52945-8519-4188-9e67-a89687a072ae admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:35.230912 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f52945-8519-4188-9e67-a89687a072ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:35.232659 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f52945-8519-4188-9e67-a89687a072ae admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:35.279783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:35.291604 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 951/1910] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:34:34 2022] PUT /identity/v3/projects/48068f2b2fc243f08ae1688ed32d7bc1/users/a8bb21e782d7442e928c995fad79f2d1/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:35.292566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e57e43-d555-4bda-87eb-afd91711502d admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:35.320793 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 960/1911] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:34:35 2022] POST /identity/v3/users => generated 291 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:34:35.320793 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f52945-8519-4188-9e67-a89687a072ae admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:35.339435 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f52945-8519-4188-9e67-a89687a072ae admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:35.408071 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e57e43-d555-4bda-87eb-afd91711502d admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:35.416867 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee01fe-6d48-44dc-8c4c-432e0c29c129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:35.417710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee01fe-6d48-44dc-8c4c-432e0c29c129 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:35.418042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee01fe-6d48-44dc-8c4c-432e0c29c129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:35.418324 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee01fe-6d48-44dc-8c4c-432e0c29c129 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:35.419002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:35.423987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee01fe-6d48-44dc-8c4c-432e0c29c129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:35.423987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee01fe-6d48-44dc-8c4c-432e0c29c129 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:35.431582 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027680b-8434-4b11-a0ff-4cbbdc501788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:35.432487 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027680b-8434-4b11-a0ff-4cbbdc501788 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:35.432778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027680b-8434-4b11-a0ff-4cbbdc501788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:35.433054 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027680b-8434-4b11-a0ff-4cbbdc501788 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:35.433794 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d027680b-8434-4b11-a0ff-4cbbdc501788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:35.438983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d027680b-8434-4b11-a0ff-4cbbdc501788 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:35.439130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 952/1912] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:34:35.439130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee01fe-6d48-44dc-8c4c-432e0c29c129 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:35.451632 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 961/1913] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:34:35.451632 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027680b-8434-4b11-a0ff-4cbbdc501788 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:35.457479 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee01fe-6d48-44dc-8c4c-432e0c29c129 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:35.462033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027680b-8434-4b11-a0ff-4cbbdc501788 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:35.467259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84800c12-16aa-4a49-a56c-c2b9f81ccdd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:35.468387 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84800c12-16aa-4a49-a56c-c2b9f81ccdd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:35.468682 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84800c12-16aa-4a49-a56c-c2b9f81ccdd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:35.468963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84800c12-16aa-4a49-a56c-c2b9f81ccdd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:35.469699 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84800c12-16aa-4a49-a56c-c2b9f81ccdd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:35.471979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17fa65-512b-4864-bc71-ca0ece7e6f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:35.472775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17fa65-512b-4864-bc71-ca0ece7e6f5a admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:35.473161 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17fa65-512b-4864-bc71-ca0ece7e6f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:35.473829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae17fa65-512b-4864-bc71-ca0ece7e6f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:35.473946 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84800c12-16aa-4a49-a56c-c2b9f81ccdd8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:35.474469 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae17fa65-512b-4864-bc71-ca0ece7e6f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:35.476397 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae17fa65-512b-4864-bc71-ca0ece7e6f5a admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:35.497689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 962/1914] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:34:35.503928 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84800c12-16aa-4a49-a56c-c2b9f81ccdd8 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:35.508639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 953/1915] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:34:35.512736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17fa65-512b-4864-bc71-ca0ece7e6f5a admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:35.517251 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84800c12-16aa-4a49-a56c-c2b9f81ccdd8 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:35.524307 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae17fa65-512b-4864-bc71-ca0ece7e6f5a admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:35.524949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99866b81-c111-47da-88e0-a5eedb66ff4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:35.526036 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99866b81-c111-47da-88e0-a5eedb66ff4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:35.526494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99866b81-c111-47da-88e0-a5eedb66ff4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:35.527313 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99866b81-c111-47da-88e0-a5eedb66ff4c admin admin] PATH_INFO: `/v3/projects/48068f2b2fc243f08ae1688ed32d7bc1/users/a8bb21e782d7442e928c995fad79f2d1/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:35.529247 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99866b81-c111-47da-88e0-a5eedb66ff4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=48068f2b2fc243f08ae1688ed32d7bc1, user_id=a8bb21e782d7442e928c995fad79f2d1, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:35.531147 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf4d44-4360-4bb5-b4ef-1c3b499a4105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:35.532330 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf4d44-4360-4bb5-b4ef-1c3b499a4105 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:35.532574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99866b81-c111-47da-88e0-a5eedb66ff4c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:35.532680 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf4d44-4360-4bb5-b4ef-1c3b499a4105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:35.535139 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf4d44-4360-4bb5-b4ef-1c3b499a4105 admin admin] PATH_INFO: `/v3/projects/ebbe2271c89e4ccabd92b351d0a2ce4c/users/ac316d9eb0194942aed41a34a2f1c503/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:35.551165 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 963/1916] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:34:35 2022] PUT /identity/v3/projects/48068f2b2fc243f08ae1688ed32d7bc1/users/a8bb21e782d7442e928c995fad79f2d1/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:35.552148 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99866b81-c111-47da-88e0-a5eedb66ff4c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:35.559290 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf4d44-4360-4bb5-b4ef-1c3b499a4105 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebbe2271c89e4ccabd92b351d0a2ce4c, user_id=ac316d9eb0194942aed41a34a2f1c503, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:35.562098 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf4d44-4360-4bb5-b4ef-1c3b499a4105 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:35.624815 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99866b81-c111-47da-88e0-a5eedb66ff4c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:35.631805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 954/1917] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:34:35 2022] PUT /identity/v3/projects/ebbe2271c89e4ccabd92b351d0a2ce4c/users/ac316d9eb0194942aed41a34a2f1c503/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:35.634959 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706aa4b-ef8e-47b9-b5c9-57d83bd86453 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:35.635230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706aa4b-ef8e-47b9-b5c9-57d83bd86453 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:35.635596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f706aa4b-ef8e-47b9-b5c9-57d83bd86453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:35.677533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-f706aa4b-ef8e-47b9-b5c9-57d83bd86453 None None] MFA Rules not processed for user `a8bb21e782d7442e928c995fad79f2d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:35.678729 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-f706aa4b-ef8e-47b9-b5c9-57d83bd86453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:35.714216 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11086cf-0742-4c69-91b5-24f5787fb91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:35.714965 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11086cf-0742-4c69-91b5-24f5787fb91c admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:35.715165 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11086cf-0742-4c69-91b5-24f5787fb91c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:35.715483 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11086cf-0742-4c69-91b5-24f5787fb91c admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:35.716217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11086cf-0742-4c69-91b5-24f5787fb91c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:35.717829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11086cf-0742-4c69-91b5-24f5787fb91c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:35.731671 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 964/1918] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:35 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:34:35.731671 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d22983-adea-4b92-b3ca-5d35b6935a27 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:35.731671 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d22983-adea-4b92-b3ca-5d35b6935a27 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:35.731671 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d22983-adea-4b92-b3ca-5d35b6935a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:35.760616 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f4d22983-adea-4b92-b3ca-5d35b6935a27 None None] MFA Rules not processed for user `ac316d9eb0194942aed41a34a2f1c503`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:35.762020 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f4d22983-adea-4b92-b3ca-5d35b6935a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:35.793749 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 955/1919] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:35 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:35.795455 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f706aa4b-ef8e-47b9-b5c9-57d83bd86453 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:35.823855 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f706aa4b-ef8e-47b9-b5c9-57d83bd86453 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:35.831673 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c54c38-ee38-4581-9e3e-b48de19ccd62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:35.832964 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c54c38-ee38-4581-9e3e-b48de19ccd62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:35.833824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c54c38-ee38-4581-9e3e-b48de19ccd62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:35.834236 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c54c38-ee38-4581-9e3e-b48de19ccd62 admin admin] PATH_INFO: `/v3/projects/71be96b4ceb345128cdd75e7a5cbb4b5/users/070f588215014ef2a4fb870e74228f4a/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:35.844294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:35.853705 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c54c38-ee38-4581-9e3e-b48de19ccd62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71be96b4ceb345128cdd75e7a5cbb4b5, user_id=070f588215014ef2a4fb870e74228f4a, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:35.855915 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 965/1920] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:35.856501 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c54c38-ee38-4581-9e3e-b48de19ccd62 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:35.856946 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d22983-adea-4b92-b3ca-5d35b6935a27 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:35.860705 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f4d22983-adea-4b92-b3ca-5d35b6935a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:35.918626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d22983-adea-4b92-b3ca-5d35b6935a27 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:35.922335 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad6f87b-fd97-44c3-824b-3d7e93b4d95d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:35.922987 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6f87b-fd97-44c3-824b-3d7e93b4d95d service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:35.923290 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6f87b-fd97-44c3-824b-3d7e93b4d95d service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:35.923589 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad6f87b-fd97-44c3-824b-3d7e93b4d95d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:35.939015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6f87b-fd97-44c3-824b-3d7e93b4d95d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:35.939015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad6f87b-fd97-44c3-824b-3d7e93b4d95d service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:35.941372 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 956/1921] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:34:35 2022] PUT /identity/v3/projects/71be96b4ceb345128cdd75e7a5cbb4b5/users/070f588215014ef2a4fb870e74228f4a/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:35.942350 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c54c38-ee38-4581-9e3e-b48de19ccd62 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:35.952793 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 966/1922] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:34:35 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:34:35.954957 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80537022-d3d7-4538-976b-4ee92a4714e3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:35.956251 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80537022-d3d7-4538-976b-4ee92a4714e3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:35.956594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80537022-d3d7-4538-976b-4ee92a4714e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:35.985802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:35.995337 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-80537022-d3d7-4538-976b-4ee92a4714e3 None None] MFA Rules not processed for user `070f588215014ef2a4fb870e74228f4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:35.997193 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-80537022-d3d7-4538-976b-4ee92a4714e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:36.037166 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c54c38-ee38-4581-9e3e-b48de19ccd62 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:36.044219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79ff66-4782-4d3e-a5bd-864c4a6abb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:36.045065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79ff66-4782-4d3e-a5bd-864c4a6abb2d admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:36.045400 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79ff66-4782-4d3e-a5bd-864c4a6abb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:36.045794 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79ff66-4782-4d3e-a5bd-864c4a6abb2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:36.047530 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc79ff66-4782-4d3e-a5bd-864c4a6abb2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:36.050599 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc79ff66-4782-4d3e-a5bd-864c4a6abb2d admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:36.093466 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 967/1923] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:36.100060 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80537022-d3d7-4538-976b-4ee92a4714e3 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:36.116251 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80537022-d3d7-4538-976b-4ee92a4714e3 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:36.125886 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ca0b7-3ca0-4582-afba-16597bd11806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:36.126680 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ca0b7-3ca0-4582-afba-16597bd11806 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:36.127177 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ca0b7-3ca0-4582-afba-16597bd11806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:36.128308 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601ca0b7-3ca0-4582-afba-16597bd11806 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:36.128308 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601ca0b7-3ca0-4582-afba-16597bd11806 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:36.130352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601ca0b7-3ca0-4582-afba-16597bd11806 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:36.186465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 957/1924] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:34:35 2022] POST /identity/v3/projects => generated 383 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:34:36.187459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79ff66-4782-4d3e-a5bd-864c4a6abb2d admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:36.208235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 968/1925] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:34:36 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:34:36.213395 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ca0b7-3ca0-4582-afba-16597bd11806 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:36.267380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79ff66-4782-4d3e-a5bd-864c4a6abb2d admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:36.307369 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601ca0b7-3ca0-4582-afba-16597bd11806 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:36.313312 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4d254-b097-42eb-855c-556390f4c547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:36.314064 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4d254-b097-42eb-855c-556390f4c547 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:36.314358 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4d254-b097-42eb-855c-556390f4c547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:36.314689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4d254-b097-42eb-855c-556390f4c547 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:36.315621 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da4d254-b097-42eb-855c-556390f4c547 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:36.319240 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da4d254-b097-42eb-855c-556390f4c547 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:36.368687 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c0a17-674f-4443-9c2c-8bc9b315536b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:36.369719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c0a17-674f-4443-9c2c-8bc9b315536b admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:36.370034 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:36.370396 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c0a17-674f-4443-9c2c-8bc9b315536b admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:36.373298 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3c0a17-674f-4443-9c2c-8bc9b315536b admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:36.373298 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c0a17-674f-4443-9c2c-8bc9b315536b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:36.373598 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3c0a17-674f-4443-9c2c-8bc9b315536b admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:36.414466 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 969/1926] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:34:36 2022] POST /identity/v3/users => generated 298 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:34:36.415839 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4d254-b097-42eb-855c-556390f4c547 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:36.434563 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4d254-b097-42eb-855c-556390f4c547 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:36.450439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1339214a-4b86-4c1a-ba3d-fbf895472a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:36.451788 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339214a-4b86-4c1a-ba3d-fbf895472a75 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:36.451788 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339214a-4b86-4c1a-ba3d-fbf895472a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:36.452276 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1339214a-4b86-4c1a-ba3d-fbf895472a75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:36.453144 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1339214a-4b86-4c1a-ba3d-fbf895472a75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:36.455183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1339214a-4b86-4c1a-ba3d-fbf895472a75 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:36.474014 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 970/1927] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:34:36.475292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1339214a-4b86-4c1a-ba3d-fbf895472a75 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:36.489819 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1339214a-4b86-4c1a-ba3d-fbf895472a75 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:36.496104 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec052a98-b7b8-466e-8ff2-fc1c7e7dd697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:36.497358 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec052a98-b7b8-466e-8ff2-fc1c7e7dd697 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:36.498420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec052a98-b7b8-466e-8ff2-fc1c7e7dd697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:36.498913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec052a98-b7b8-466e-8ff2-fc1c7e7dd697 admin admin] PATH_INFO: `/v3/projects/2fb3bf97dd004cf9b9b7e89316bd6b39/users/3792b9715fa6412fb32a800942a62ce5/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:36.511918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 958/1928] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:34:36 2022] POST /identity/v3/users => generated 298 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:34:36.512700 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c0a17-674f-4443-9c2c-8bc9b315536b admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:36.523003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3c0a17-674f-4443-9c2c-8bc9b315536b admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:36.528801 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6e93e-da24-40e4-a19e-67ee21da1035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:36.529842 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6e93e-da24-40e4-a19e-67ee21da1035 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:36.530135 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6e93e-da24-40e4-a19e-67ee21da1035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:36.530408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b6e93e-da24-40e4-a19e-67ee21da1035 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:36.534008 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b6e93e-da24-40e4-a19e-67ee21da1035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:36.534008 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b6e93e-da24-40e4-a19e-67ee21da1035 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:36.542186 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec052a98-b7b8-466e-8ff2-fc1c7e7dd697 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb3bf97dd004cf9b9b7e89316bd6b39, user_id=3792b9715fa6412fb32a800942a62ce5, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:36.559595 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 959/1929] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:34:36.560799 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6e93e-da24-40e4-a19e-67ee21da1035 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:36.563490 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec052a98-b7b8-466e-8ff2-fc1c7e7dd697 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:36.581665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b6e93e-da24-40e4-a19e-67ee21da1035 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:36.588288 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996e224-3d7c-40bd-b3f9-3ff65c157f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:36.589422 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996e224-3d7c-40bd-b3f9-3ff65c157f7b admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:36.589898 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996e224-3d7c-40bd-b3f9-3ff65c157f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:36.590918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f996e224-3d7c-40bd-b3f9-3ff65c157f7b admin admin] PATH_INFO: `/v3/projects/8bae28169f164612835d8123bdeb2744/users/46eff453481747aa91323d3c3beda651/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:36.613027 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f996e224-3d7c-40bd-b3f9-3ff65c157f7b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bae28169f164612835d8123bdeb2744, user_id=46eff453481747aa91323d3c3beda651, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:36.615902 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f996e224-3d7c-40bd-b3f9-3ff65c157f7b admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:36.654667 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 971/1930] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:34:36 2022] PUT /identity/v3/projects/2fb3bf97dd004cf9b9b7e89316bd6b39/users/3792b9715fa6412fb32a800942a62ce5/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:36.658626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec052a98-b7b8-466e-8ff2-fc1c7e7dd697 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:36.678534 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 960/1931] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:34:36 2022] PUT /identity/v3/projects/8bae28169f164612835d8123bdeb2744/users/46eff453481747aa91323d3c3beda651/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:36.681845 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996e224-3d7c-40bd-b3f9-3ff65c157f7b admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:36.693982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:36.748558 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec052a98-b7b8-466e-8ff2-fc1c7e7dd697 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:36.753643 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33863370-eb53-4f8a-a112-1105a80bcc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:36.754327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33863370-eb53-4f8a-a112-1105a80bcc36 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:36.754588 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f996e224-3d7c-40bd-b3f9-3ff65c157f7b admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:36.754689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33863370-eb53-4f8a-a112-1105a80bcc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:36.754893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33863370-eb53-4f8a-a112-1105a80bcc36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:36.755759 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33863370-eb53-4f8a-a112-1105a80bcc36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:36.757436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33863370-eb53-4f8a-a112-1105a80bcc36 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:36.760408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357f8a5-0bf8-4b47-a227-fa2049003844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:36.761086 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357f8a5-0bf8-4b47-a227-fa2049003844 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:36.761373 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357f8a5-0bf8-4b47-a227-fa2049003844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:36.761652 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b357f8a5-0bf8-4b47-a227-fa2049003844 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:36.762388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b357f8a5-0bf8-4b47-a227-fa2049003844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:36.764182 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b357f8a5-0bf8-4b47-a227-fa2049003844 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:36.771200 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 972/1932] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:34:36.775884 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33863370-eb53-4f8a-a112-1105a80bcc36 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:36.785688 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 961/1933] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:34:36.788932 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357f8a5-0bf8-4b47-a227-fa2049003844 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:36.789880 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33863370-eb53-4f8a-a112-1105a80bcc36 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:36.795333 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a53850-a007-4773-9a69-38b155b71a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:36.796372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a53850-a007-4773-9a69-38b155b71a21 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:36.796702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a53850-a007-4773-9a69-38b155b71a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:36.796988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a53850-a007-4773-9a69-38b155b71a21 admin admin] PATH_INFO: `/v3/projects/2fb3bf97dd004cf9b9b7e89316bd6b39/users/3792b9715fa6412fb32a800942a62ce5/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:36.799134 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a53850-a007-4773-9a69-38b155b71a21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb3bf97dd004cf9b9b7e89316bd6b39, user_id=3792b9715fa6412fb32a800942a62ce5, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:36.802142 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a53850-a007-4773-9a69-38b155b71a21 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:36.803021 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b357f8a5-0bf8-4b47-a227-fa2049003844 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:36.810442 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63a9ccc-d0cc-4d2a-ad5c-0844435e18e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:36.811411 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a9ccc-d0cc-4d2a-ad5c-0844435e18e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:36.812337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a9ccc-d0cc-4d2a-ad5c-0844435e18e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:36.812844 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63a9ccc-d0cc-4d2a-ad5c-0844435e18e4 admin admin] PATH_INFO: `/v3/projects/8bae28169f164612835d8123bdeb2744/users/46eff453481747aa91323d3c3beda651/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:36.815126 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a9ccc-d0cc-4d2a-ad5c-0844435e18e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bae28169f164612835d8123bdeb2744, user_id=46eff453481747aa91323d3c3beda651, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:36.817925 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63a9ccc-d0cc-4d2a-ad5c-0844435e18e4 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:36.829556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 973/1934] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:34:36 2022] PUT /identity/v3/projects/2fb3bf97dd004cf9b9b7e89316bd6b39/users/3792b9715fa6412fb32a800942a62ce5/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:36.831492 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c1a19-0751-47a3-aa0d-5b2c417c7241 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:36.831874 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c1a19-0751-47a3-aa0d-5b2c417c7241 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:36.832360 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7c1a19-0751-47a3-aa0d-5b2c417c7241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:36.832468 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 962/1935] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:34:36 2022] PUT /identity/v3/projects/8bae28169f164612835d8123bdeb2744/users/46eff453481747aa91323d3c3beda651/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:34:36.842557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b094d5e-67d6-482a-83a7-3933885402da None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:36.842854 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b094d5e-67d6-482a-83a7-3933885402da None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:36.843136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b094d5e-67d6-482a-83a7-3933885402da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:36.866080 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5a7c1a19-0751-47a3-aa0d-5b2c417c7241 None None] MFA Rules not processed for user `3792b9715fa6412fb32a800942a62ce5`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:36.867876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5a7c1a19-0751-47a3-aa0d-5b2c417c7241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:36.876214 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-2b094d5e-67d6-482a-83a7-3933885402da None None] MFA Rules not processed for user `46eff453481747aa91323d3c3beda651`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:36.878225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-2b094d5e-67d6-482a-83a7-3933885402da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:36.966030 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 974/1936] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:34:36.979455 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c1a19-0751-47a3-aa0d-5b2c417c7241 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:36.987755 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 963/1937] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:34:37.014243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:37.038504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7c1a19-0751-47a3-aa0d-5b2c417c7241 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:37.043290 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4778991-acad-4d78-93fe-4762fc650ecd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:37.043955 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4778991-acad-4d78-93fe-4762fc650ecd service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:37.044237 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4778991-acad-4d78-93fe-4762fc650ecd service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:37.044510 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4778991-acad-4d78-93fe-4762fc650ecd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:37.053993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4778991-acad-4d78-93fe-4762fc650ecd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:37.056243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4778991-acad-4d78-93fe-4762fc650ecd service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:37.065891 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 975/1938] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:34:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:34:37.093865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b094d5e-67d6-482a-83a7-3933885402da None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:37.095637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-2b094d5e-67d6-482a-83a7-3933885402da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:37.109882 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b094d5e-67d6-482a-83a7-3933885402da None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:37.114131 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa525d-beb7-4306-82cc-193120aa0c28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:37.114885 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa525d-beb7-4306-82cc-193120aa0c28 service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:37.115850 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa525d-beb7-4306-82cc-193120aa0c28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:37.115850 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa525d-beb7-4306-82cc-193120aa0c28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:37.193481 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa525d-beb7-4306-82cc-193120aa0c28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:37.194999 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afa525d-beb7-4306-82cc-193120aa0c28 service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:37.206571 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 964/1939] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:34:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:34:37.576082 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4778991-acad-4d78-93fe-4762fc650ecd service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:37.585720 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4778991-acad-4d78-93fe-4762fc650ecd service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:37.590333 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732406fa-c9fc-48b1-a61d-405a162ab8a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:37.590979 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732406fa-c9fc-48b1-a61d-405a162ab8a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:37.591252 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732406fa-c9fc-48b1-a61d-405a162ab8a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:37.591554 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732406fa-c9fc-48b1-a61d-405a162ab8a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:37.678282 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732406fa-c9fc-48b1-a61d-405a162ab8a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:37.680863 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732406fa-c9fc-48b1-a61d-405a162ab8a2 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:37.692695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 976/1940] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:34:37 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:34:38.287839 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa525d-beb7-4306-82cc-193120aa0c28 service cinder] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:38.302281 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:38.355090 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa525d-beb7-4306-82cc-193120aa0c28 service cinder] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:38.361659 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41066b-ac93-435b-8fb7-e6cfc61d6c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:38.362592 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41066b-ac93-435b-8fb7-e6cfc61d6c41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:38.362881 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41066b-ac93-435b-8fb7-e6cfc61d6c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:38.363160 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b41066b-ac93-435b-8fb7-e6cfc61d6c41 admin admin] PATH_INFO: `/v3/users/070f588215014ef2a4fb870e74228f4a` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:38.364440 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b41066b-ac93-435b-8fb7-e6cfc61d6c41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=070f588215014ef2a4fb870e74228f4a)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:38.366528 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b41066b-ac93-435b-8fb7-e6cfc61d6c41 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:38.530993 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2b41066b-ac93-435b-8fb7-e6cfc61d6c41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '070f588215014ef2a4fb870e74228f4a'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:38.598634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2b41066b-ac93-435b-8fb7-e6cfc61d6c41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '070f588215014ef2a4fb870e74228f4a'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:38.604586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2b41066b-ac93-435b-8fb7-e6cfc61d6c41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '070f588215014ef2a4fb870e74228f4a'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:38.605278 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2b41066b-ac93-435b-8fb7-e6cfc61d6c41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '070f588215014ef2a4fb870e74228f4a'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:38.605914 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2b41066b-ac93-435b-8fb7-e6cfc61d6c41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '070f588215014ef2a4fb870e74228f4a'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:38.618613 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 965/1941] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:34:38 2022] DELETE /identity/v3/users/070f588215014ef2a4fb870e74228f4a => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:38.624562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ed850-95f6-4572-8e27-a4840040e5b5 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:38.625022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ed850-95f6-4572-8e27-a4840040e5b5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:38.625157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ed850-95f6-4572-8e27-a4840040e5b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:38.644095 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-964ed850-95f6-4572-8e27-a4840040e5b5 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:38.645318 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-964ed850-95f6-4572-8e27-a4840040e5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:38.713088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 977/1942] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:38.722889 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41066b-ac93-435b-8fb7-e6cfc61d6c41 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:38.724343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-2b41066b-ac93-435b-8fb7-e6cfc61d6c41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:38.781255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b41066b-ac93-435b-8fb7-e6cfc61d6c41 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:38.785333 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98236b1a-ad11-4b4a-a7df-01b363cebdc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:38.785984 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98236b1a-ad11-4b4a-a7df-01b363cebdc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:38.786270 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98236b1a-ad11-4b4a-a7df-01b363cebdc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:38.786547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98236b1a-ad11-4b4a-a7df-01b363cebdc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:38.797729 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98236b1a-ad11-4b4a-a7df-01b363cebdc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:38.799164 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98236b1a-ad11-4b4a-a7df-01b363cebdc4 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:38.809950 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 966/1943] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:34:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:34:39.359583 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ed850-95f6-4572-8e27-a4840040e5b5 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:39.360960 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-964ed850-95f6-4572-8e27-a4840040e5b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:39.375462 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ed850-95f6-4572-8e27-a4840040e5b5 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:39.384372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eca91c-5e40-4154-b6e2-cba4b8e3a9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:39.385299 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eca91c-5e40-4154-b6e2-cba4b8e3a9fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:39.385586 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eca91c-5e40-4154-b6e2-cba4b8e3a9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:39.385865 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eca91c-5e40-4154-b6e2-cba4b8e3a9fd admin admin] PATH_INFO: `/v3/projects/71be96b4ceb345128cdd75e7a5cbb4b5` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:39.387283 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eca91c-5e40-4154-b6e2-cba4b8e3a9fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=71be96b4ceb345128cdd75e7a5cbb4b5)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:39.389376 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eca91c-5e40-4154-b6e2-cba4b8e3a9fd admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:39.509976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-96eca91c-5e40-4154-b6e2-cba4b8e3a9fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71be96b4ceb345128cdd75e7a5cbb4b5', 'request_id': 'req-96eca91c-5e40-4154-b6e2-cba4b8e3a9fd'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:39.601669 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-96eca91c-5e40-4154-b6e2-cba4b8e3a9fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71be96b4ceb345128cdd75e7a5cbb4b5', 'request_id': 'req-96eca91c-5e40-4154-b6e2-cba4b8e3a9fd'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:39.608330 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-96eca91c-5e40-4154-b6e2-cba4b8e3a9fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71be96b4ceb345128cdd75e7a5cbb4b5', 'request_id': 'req-96eca91c-5e40-4154-b6e2-cba4b8e3a9fd'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:39.609457 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:39.622032 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-96eca91c-5e40-4154-b6e2-cba4b8e3a9fd admin admin] The token cache is being invalidate because project 71be96b4ceb345128cdd75e7a5cbb4b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:34:39.622420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-96eca91c-5e40-4154-b6e2-cba4b8e3a9fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:39.623259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-96eca91c-5e40-4154-b6e2-cba4b8e3a9fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:39.625121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 978/1944] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:34:39 2022] DELETE /identity/v3/projects/71be96b4ceb345128cdd75e7a5cbb4b5 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:39.629000 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98236b1a-ad11-4b4a-a7df-01b363cebdc4 service neutron] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:39.639632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98236b1a-ad11-4b4a-a7df-01b363cebdc4 service neutron] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:39.665120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eca91c-5e40-4154-b6e2-cba4b8e3a9fd admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:39.666557 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-96eca91c-5e40-4154-b6e2-cba4b8e3a9fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:39.673691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:39.700034 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e22cd96-765b-4259-86ec-7e2eace7c4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:39.701065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22cd96-765b-4259-86ec-7e2eace7c4a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:39.701352 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22cd96-765b-4259-86ec-7e2eace7c4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:39.701679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e22cd96-765b-4259-86ec-7e2eace7c4a6 admin admin] PATH_INFO: `/v3/users/3792b9715fa6412fb32a800942a62ce5` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:39.702888 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e22cd96-765b-4259-86ec-7e2eace7c4a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3792b9715fa6412fb32a800942a62ce5)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:39.704923 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e22cd96-765b-4259-86ec-7e2eace7c4a6 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:39.734339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eca91c-5e40-4154-b6e2-cba4b8e3a9fd admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:39.738961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5016b2-5db7-4338-9fa4-49f3740f2944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:39.739860 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5016b2-5db7-4338-9fa4-49f3740f2944 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:39.740142 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5016b2-5db7-4338-9fa4-49f3740f2944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:39.740417 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5016b2-5db7-4338-9fa4-49f3740f2944 admin admin] PATH_INFO: `/v3/users/4453d4ca6665403292f6cbfbf57d9cae` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:39.741603 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5016b2-5db7-4338-9fa4-49f3740f2944 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4453d4ca6665403292f6cbfbf57d9cae)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:39.744086 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5016b2-5db7-4338-9fa4-49f3740f2944 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:39.918730 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2e22cd96-765b-4259-86ec-7e2eace7c4a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3792b9715fa6412fb32a800942a62ce5'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:39.948084 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2c5016b2-5db7-4338-9fa4-49f3740f2944 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4453d4ca6665403292f6cbfbf57d9cae'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:39.948742 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2c5016b2-5db7-4338-9fa4-49f3740f2944 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4453d4ca6665403292f6cbfbf57d9cae'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:39.955706 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2c5016b2-5db7-4338-9fa4-49f3740f2944 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4453d4ca6665403292f6cbfbf57d9cae'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:39.956399 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2c5016b2-5db7-4338-9fa4-49f3740f2944 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4453d4ca6665403292f6cbfbf57d9cae'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:39.979380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2e22cd96-765b-4259-86ec-7e2eace7c4a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3792b9715fa6412fb32a800942a62ce5'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:39.991449 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2e22cd96-765b-4259-86ec-7e2eace7c4a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3792b9715fa6412fb32a800942a62ce5'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:39.992171 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2e22cd96-765b-4259-86ec-7e2eace7c4a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3792b9715fa6412fb32a800942a62ce5'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:39.992725 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2e22cd96-765b-4259-86ec-7e2eace7c4a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3792b9715fa6412fb32a800942a62ce5'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:40.009851 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 967/1945] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:34:39 2022] DELETE /identity/v3/users/3792b9715fa6412fb32a800942a62ce5 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:40.076485 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2c5016b2-5db7-4338-9fa4-49f3740f2944 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4453d4ca6665403292f6cbfbf57d9cae'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:40.097039 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 979/1946] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:34:39 2022] DELETE /identity/v3/users/4453d4ca6665403292f6cbfbf57d9cae => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:34:40.101755 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907037ef-b621-4247-a6e8-c1e831ac7af1 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:40.102082 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907037ef-b621-4247-a6e8-c1e831ac7af1 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:40.102365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907037ef-b621-4247-a6e8-c1e831ac7af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:40.120526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-907037ef-b621-4247-a6e8-c1e831ac7af1 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:40.121737 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-907037ef-b621-4247-a6e8-c1e831ac7af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:40.185485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 968/1947] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:40.198825 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5016b2-5db7-4338-9fa4-49f3740f2944 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:40.200425 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2c5016b2-5db7-4338-9fa4-49f3740f2944 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:40.261566 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5016b2-5db7-4338-9fa4-49f3740f2944 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:40.265337 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08d6ff-6cd4-4928-8579-2d4484ccd7c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:40.266274 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08d6ff-6cd4-4928-8579-2d4484ccd7c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:40.266562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08d6ff-6cd4-4928-8579-2d4484ccd7c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:40.266842 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b08d6ff-6cd4-4928-8579-2d4484ccd7c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:40.277393 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08d6ff-6cd4-4928-8579-2d4484ccd7c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:40.278841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b08d6ff-6cd4-4928-8579-2d4484ccd7c3 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:40.289506 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 980/1948] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:34:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:34:40.591603 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907037ef-b621-4247-a6e8-c1e831ac7af1 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:40.593078 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-907037ef-b621-4247-a6e8-c1e831ac7af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:40.606952 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907037ef-b621-4247-a6e8-c1e831ac7af1 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:40.612386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64935df-8cf9-4200-b1f2-3c205f4164ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:40.613245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64935df-8cf9-4200-b1f2-3c205f4164ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:40.613525 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64935df-8cf9-4200-b1f2-3c205f4164ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:40.613826 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64935df-8cf9-4200-b1f2-3c205f4164ff admin admin] PATH_INFO: `/v3/projects/2fb3bf97dd004cf9b9b7e89316bd6b39` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:40.615195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64935df-8cf9-4200-b1f2-3c205f4164ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fb3bf97dd004cf9b9b7e89316bd6b39)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:40.617653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64935df-8cf9-4200-b1f2-3c205f4164ff admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:40.741298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08d6ff-6cd4-4928-8579-2d4484ccd7c3 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:40.743704 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2b08d6ff-6cd4-4928-8579-2d4484ccd7c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:40.754725 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-b64935df-8cf9-4200-b1f2-3c205f4164ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fb3bf97dd004cf9b9b7e89316bd6b39', 'request_id': 'req-b64935df-8cf9-4200-b1f2-3c205f4164ff'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:40.756079 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:40.761725 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b08d6ff-6cd4-4928-8579-2d4484ccd7c3 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:40.763707 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-b64935df-8cf9-4200-b1f2-3c205f4164ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fb3bf97dd004cf9b9b7e89316bd6b39', 'request_id': 'req-b64935df-8cf9-4200-b1f2-3c205f4164ff'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:40.770682 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-b64935df-8cf9-4200-b1f2-3c205f4164ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fb3bf97dd004cf9b9b7e89316bd6b39', 'request_id': 'req-b64935df-8cf9-4200-b1f2-3c205f4164ff'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:40.777722 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:40.823807 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fd09b-a8ac-478a-85c3-0e91e2333561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:40.824753 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fd09b-a8ac-478a-85c3-0e91e2333561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:40.825037 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fd09b-a8ac-478a-85c3-0e91e2333561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:40.825321 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fd09b-a8ac-478a-85c3-0e91e2333561 admin admin] PATH_INFO: `/v3/projects/d9d9b6a7fb7846a0874e86f7e20da02e` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:40.826501 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817fd09b-a8ac-478a-85c3-0e91e2333561 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9d9b6a7fb7846a0874e86f7e20da02e)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:40.828461 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817fd09b-a8ac-478a-85c3-0e91e2333561 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:40.871639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-b64935df-8cf9-4200-b1f2-3c205f4164ff admin admin] The token cache is being invalidate because project 2fb3bf97dd004cf9b9b7e89316bd6b39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:34:40.871926 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-b64935df-8cf9-4200-b1f2-3c205f4164ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:40.872575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-b64935df-8cf9-4200-b1f2-3c205f4164ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:40.874115 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 969/1949] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:34:40 2022] DELETE /identity/v3/projects/2fb3bf97dd004cf9b9b7e89316bd6b39 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:40.878733 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6ee0c-b18e-4ddc-8c70-dda8a0cd2f2b None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:40.879018 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6ee0c-b18e-4ddc-8c70-dda8a0cd2f2b None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:40.879318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f6ee0c-b18e-4ddc-8c70-dda8a0cd2f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:40.898094 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-96f6ee0c-b18e-4ddc-8c70-dda8a0cd2f2b None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:40.899487 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-96f6ee0c-b18e-4ddc-8c70-dda8a0cd2f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:40.907786 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 970/1950] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:34:40.915734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf6f8b-d675-49da-9099-526a03fd88a0 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:40.916021 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf6f8b-d675-49da-9099-526a03fd88a0 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:40.916299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bf6f8b-d675-49da-9099-526a03fd88a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:40.935586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-53bf6f8b-d675-49da-9099-526a03fd88a0 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:40.936791 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-53bf6f8b-d675-49da-9099-526a03fd88a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:40.940000 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-817fd09b-a8ac-478a-85c3-0e91e2333561 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9d9b6a7fb7846a0874e86f7e20da02e', 'request_id': 'req-817fd09b-a8ac-478a-85c3-0e91e2333561'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:41.001006 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 971/1951] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:34:41.008392 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf6f8b-d675-49da-9099-526a03fd88a0 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:41.017552 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bf6f8b-d675-49da-9099-526a03fd88a0 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:41.022430 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee0f29-7a35-45b4-abfb-6ace29f6adf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:41.023081 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee0f29-7a35-45b4-abfb-6ace29f6adf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:41.023583 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee0f29-7a35-45b4-abfb-6ace29f6adf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:41.024237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee0f29-7a35-45b4-abfb-6ace29f6adf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:41.024967 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee0f29-7a35-45b4-abfb-6ace29f6adf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:34:41.025280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee0f29-7a35-45b4-abfb-6ace29f6adf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:41.025813 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-817fd09b-a8ac-478a-85c3-0e91e2333561 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9d9b6a7fb7846a0874e86f7e20da02e', 'request_id': 'req-817fd09b-a8ac-478a-85c3-0e91e2333561'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:41.026878 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee0f29-7a35-45b4-abfb-6ace29f6adf8 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:41.032502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-817fd09b-a8ac-478a-85c3-0e91e2333561 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9d9b6a7fb7846a0874e86f7e20da02e', 'request_id': 'req-817fd09b-a8ac-478a-85c3-0e91e2333561'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:41.039428 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 972/1952] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:34:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:34:41.044277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee0f29-7a35-45b4-abfb-6ace29f6adf8 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:41.044410 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-817fd09b-a8ac-478a-85c3-0e91e2333561 admin admin] The token cache is being invalidate because project d9d9b6a7fb7846a0874e86f7e20da02e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:34:41.044599 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-817fd09b-a8ac-478a-85c3-0e91e2333561 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:41.045331 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-817fd09b-a8ac-478a-85c3-0e91e2333561 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:41.046903 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 981/1953] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:34:40 2022] DELETE /identity/v3/projects/d9d9b6a7fb7846a0874e86f7e20da02e => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:41.050530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fd09b-a8ac-478a-85c3-0e91e2333561 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:41.051855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-817fd09b-a8ac-478a-85c3-0e91e2333561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:41.055307 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee0f29-7a35-45b4-abfb-6ace29f6adf8 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:41.060262 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5cbb7d-6bd2-455f-a3a1-3ac807dbb746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:41.061035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5cbb7d-6bd2-455f-a3a1-3ac807dbb746 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:41.061325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5cbb7d-6bd2-455f-a3a1-3ac807dbb746 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:41.061610 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5cbb7d-6bd2-455f-a3a1-3ac807dbb746 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:41.062310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5cbb7d-6bd2-455f-a3a1-3ac807dbb746 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:41.064283 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5cbb7d-6bd2-455f-a3a1-3ac807dbb746 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:41.077305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fd09b-a8ac-478a-85c3-0e91e2333561 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:41.077305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163697d-d4f1-47c5-a8d7-99e05a3e823d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:41.077305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163697d-d4f1-47c5-a8d7-99e05a3e823d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:41.077305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163697d-d4f1-47c5-a8d7-99e05a3e823d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:41.078335 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163697d-d4f1-47c5-a8d7-99e05a3e823d admin admin] PATH_INFO: `/v3/users/a8bb21e782d7442e928c995fad79f2d1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:41.078335 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163697d-d4f1-47c5-a8d7-99e05a3e823d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8bb21e782d7442e928c995fad79f2d1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:41.078335 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163697d-d4f1-47c5-a8d7-99e05a3e823d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:41.129966 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 973/1954] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:34:41 2022] POST /identity/v3/projects => generated 375 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:34:41.134223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5cbb7d-6bd2-455f-a3a1-3ac807dbb746 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:41.135816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-bb5cbb7d-6bd2-455f-a3a1-3ac807dbb746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:41.198723 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5cbb7d-6bd2-455f-a3a1-3ac807dbb746 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:41.203639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce7c1e-b362-468b-817e-229efe20ae72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:41.204377 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce7c1e-b362-468b-817e-229efe20ae72 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:41.204659 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce7c1e-b362-468b-817e-229efe20ae72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:41.204933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ce7c1e-b362-468b-817e-229efe20ae72 admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:41.205618 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce7c1e-b362-468b-817e-229efe20ae72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:41.207559 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ce7c1e-b362-468b-817e-229efe20ae72 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:41.239590 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9163697d-d4f1-47c5-a8d7-99e05a3e823d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8bb21e782d7442e928c995fad79f2d1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:41.240256 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9163697d-d4f1-47c5-a8d7-99e05a3e823d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8bb21e782d7442e928c995fad79f2d1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:41.247635 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9163697d-d4f1-47c5-a8d7-99e05a3e823d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8bb21e782d7442e928c995fad79f2d1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:41.248289 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9163697d-d4f1-47c5-a8d7-99e05a3e823d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8bb21e782d7442e928c995fad79f2d1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:41.323224 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 974/1955] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:34:41 2022] POST /identity/v3/users => generated 288 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:34:41.327218 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce7c1e-b362-468b-817e-229efe20ae72 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:41.328969 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-13ce7c1e-b362-468b-817e-229efe20ae72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:41.342198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ce7c1e-b362-468b-817e-229efe20ae72 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:41.347521 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea23f1-37ca-4966-b4d7-e02a2fa4bb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:41.348199 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea23f1-37ca-4966-b4d7-e02a2fa4bb97 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:41.348488 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea23f1-37ca-4966-b4d7-e02a2fa4bb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:41.348770 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ea23f1-37ca-4966-b4d7-e02a2fa4bb97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:41.349533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea23f1-37ca-4966-b4d7-e02a2fa4bb97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:41.351140 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ea23f1-37ca-4966-b4d7-e02a2fa4bb97 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:41.365381 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 975/1956] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:34:41.366427 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea23f1-37ca-4966-b4d7-e02a2fa4bb97 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:41.378946 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ea23f1-37ca-4966-b4d7-e02a2fa4bb97 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:41.385545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7abba3-e267-4971-8c20-c38629b2b385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:41.386582 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7abba3-e267-4971-8c20-c38629b2b385 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:41.386870 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7abba3-e267-4971-8c20-c38629b2b385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:41.387142 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7abba3-e267-4971-8c20-c38629b2b385 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:41.387939 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7abba3-e267-4971-8c20-c38629b2b385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:41.389735 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7abba3-e267-4971-8c20-c38629b2b385 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:41.391112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9163697d-d4f1-47c5-a8d7-99e05a3e823d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8bb21e782d7442e928c995fad79f2d1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:41.403960 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 976/1957] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:41 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:34:41.419896 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7abba3-e267-4971-8c20-c38629b2b385 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:41.441206 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7abba3-e267-4971-8c20-c38629b2b385 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:41.445725 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 982/1958] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:34:41 2022] DELETE /identity/v3/users/a8bb21e782d7442e928c995fad79f2d1 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:34:41.536338 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:41.543309 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9f4ff-b579-47f3-9a29-8c407aadfbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:41.544232 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9f4ff-b579-47f3-9a29-8c407aadfbe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:41.544511 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9f4ff-b579-47f3-9a29-8c407aadfbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:41.544817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9f4ff-b579-47f3-9a29-8c407aadfbe2 admin admin] PATH_INFO: `/v3/projects/a4306927334d4345a57f0fec06893ea6/users/7e2a5236805a4ae9827334ad029c2b8b/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:41.566815 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae9f4ff-b579-47f3-9a29-8c407aadfbe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4306927334d4345a57f0fec06893ea6, user_id=7e2a5236805a4ae9827334ad029c2b8b, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:41.571231 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae9f4ff-b579-47f3-9a29-8c407aadfbe2 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:41.722853 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 977/1959] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:34:41 2022] PUT /identity/v3/projects/a4306927334d4345a57f0fec06893ea6/users/7e2a5236805a4ae9827334ad029c2b8b/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:41.733679 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd8233-e6bf-4e62-a3ad-6e825292864c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:41.733977 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd8233-e6bf-4e62-a3ad-6e825292864c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:41.734261 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd8233-e6bf-4e62-a3ad-6e825292864c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:41.762270 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d0fd8233-e6bf-4e62-a3ad-6e825292864c None None] MFA Rules not processed for user `7e2a5236805a4ae9827334ad029c2b8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:41.763590 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d0fd8233-e6bf-4e62-a3ad-6e825292864c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:41.769623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:41.859005 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 983/1960] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:41.869783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9f4ff-b579-47f3-9a29-8c407aadfbe2 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:41.871755 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-dae9f4ff-b579-47f3-9a29-8c407aadfbe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:41.931821 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9f4ff-b579-47f3-9a29-8c407aadfbe2 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:41.935608 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81964f45-61f8-4a6b-b442-f49c6069904a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:41.936263 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81964f45-61f8-4a6b-b442-f49c6069904a service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:41.936550 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81964f45-61f8-4a6b-b442-f49c6069904a service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:41.936829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81964f45-61f8-4a6b-b442-f49c6069904a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:41.945606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81964f45-61f8-4a6b-b442-f49c6069904a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:41.947079 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81964f45-61f8-4a6b-b442-f49c6069904a service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:41.956723 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 978/1961] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:34:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:34:42.058533 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd8233-e6bf-4e62-a3ad-6e825292864c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:42.060743 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d0fd8233-e6bf-4e62-a3ad-6e825292864c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:42.148440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd8233-e6bf-4e62-a3ad-6e825292864c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:42.153373 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44343874-133d-4512-820f-1d943d73a908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:42.154242 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44343874-133d-4512-820f-1d943d73a908 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:42.154848 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44343874-133d-4512-820f-1d943d73a908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:42.155059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44343874-133d-4512-820f-1d943d73a908 admin admin] PATH_INFO: `/v3/projects/48068f2b2fc243f08ae1688ed32d7bc1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:42.156288 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44343874-133d-4512-820f-1d943d73a908 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48068f2b2fc243f08ae1688ed32d7bc1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:42.158254 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44343874-133d-4512-820f-1d943d73a908 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:42.284711 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-44343874-133d-4512-820f-1d943d73a908 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48068f2b2fc243f08ae1688ed32d7bc1', 'request_id': 'req-44343874-133d-4512-820f-1d943d73a908'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:42.379474 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-44343874-133d-4512-820f-1d943d73a908 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48068f2b2fc243f08ae1688ed32d7bc1', 'request_id': 'req-44343874-133d-4512-820f-1d943d73a908'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:42.391679 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-44343874-133d-4512-820f-1d943d73a908 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48068f2b2fc243f08ae1688ed32d7bc1', 'request_id': 'req-44343874-133d-4512-820f-1d943d73a908'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:42.409904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-44343874-133d-4512-820f-1d943d73a908 admin admin] The token cache is being invalidate because project 48068f2b2fc243f08ae1688ed32d7bc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:34:42.409904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-44343874-133d-4512-820f-1d943d73a908 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:42.415564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-44343874-133d-4512-820f-1d943d73a908 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:42.416842 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 984/1962] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:34:42 2022] DELETE /identity/v3/projects/48068f2b2fc243f08ae1688ed32d7bc1 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:42.421934 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c3c5a-0ec4-446f-97b9-a07ab19a9ab3 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:42.422262 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c3c5a-0ec4-446f-97b9-a07ab19a9ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:42.422547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c3c5a-0ec4-446f-97b9-a07ab19a9ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:42.449564 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-790c3c5a-0ec4-446f-97b9-a07ab19a9ab3 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:42.456039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-790c3c5a-0ec4-446f-97b9-a07ab19a9ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:42.569310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 979/1963] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:42.582730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc3592-9b1d-41dc-8dfc-247a026c6d0c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:42.587869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc3592-9b1d-41dc-8dfc-247a026c6d0c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:42.587869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc3592-9b1d-41dc-8dfc-247a026c6d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:42.627676 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6adc3592-9b1d-41dc-8dfc-247a026c6d0c None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:42.630791 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6adc3592-9b1d-41dc-8dfc-247a026c6d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:42.652707 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 985/1964] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:42.657213 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c3c5a-0ec4-446f-97b9-a07ab19a9ab3 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:42.672864 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c3c5a-0ec4-446f-97b9-a07ab19a9ab3 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:42.681022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc3592-9b1d-41dc-8dfc-247a026c6d0c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:42.682455 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6adc3592-9b1d-41dc-8dfc-247a026c6d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:42.691840 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43258571-ba24-4f1b-99a7-fdca9a18758b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:42.692608 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43258571-ba24-4f1b-99a7-fdca9a18758b admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:42.692873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43258571-ba24-4f1b-99a7-fdca9a18758b admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:42.693149 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43258571-ba24-4f1b-99a7-fdca9a18758b admin admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:42.695507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43258571-ba24-4f1b-99a7-fdca9a18758b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:34:42.695859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43258571-ba24-4f1b-99a7-fdca9a18758b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:42.700168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43258571-ba24-4f1b-99a7-fdca9a18758b admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:42.736277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 980/1965] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:34:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:34:42.737110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43258571-ba24-4f1b-99a7-fdca9a18758b admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:42.749370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43258571-ba24-4f1b-99a7-fdca9a18758b admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:42.754377 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10eeb15-4d3d-4ecf-8c1e-df9495831f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:42.758496 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10eeb15-4d3d-4ecf-8c1e-df9495831f6c admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:42.758496 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10eeb15-4d3d-4ecf-8c1e-df9495831f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:42.758496 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10eeb15-4d3d-4ecf-8c1e-df9495831f6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:42.758496 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10eeb15-4d3d-4ecf-8c1e-df9495831f6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:42.758927 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10eeb15-4d3d-4ecf-8c1e-df9495831f6c admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:42.782162 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:42.914945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc3592-9b1d-41dc-8dfc-247a026c6d0c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:42.914945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc545d-463d-4632-9284-a96d61058df5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:42.914945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc545d-463d-4632-9284-a96d61058df5 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:42.914945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc545d-463d-4632-9284-a96d61058df5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:42.918293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cc545d-463d-4632-9284-a96d61058df5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:42.918293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f4cc545d-463d-4632-9284-a96d61058df5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:42.918293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc545d-463d-4632-9284-a96d61058df5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:42.918293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cc545d-463d-4632-9284-a96d61058df5 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:42.918293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 986/1966] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:34:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:34:43.079802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 981/1967] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:34:42 2022] POST /identity/v3/projects => generated 373 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:34:43.084121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc545d-463d-4632-9284-a96d61058df5 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:43.095915 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cc545d-463d-4632-9284-a96d61058df5 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:43.174486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af2193-bf27-4d1c-a45b-860c1ac47a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:43.175080 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af2193-bf27-4d1c-a45b-860c1ac47a09 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:43.175391 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af2193-bf27-4d1c-a45b-860c1ac47a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:43.176157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af2193-bf27-4d1c-a45b-860c1ac47a09 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:43.176870 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af2193-bf27-4d1c-a45b-860c1ac47a09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:43.178880 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af2193-bf27-4d1c-a45b-860c1ac47a09 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:43.254477 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:43.270551 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 987/1968] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:34:43 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:34:43.271594 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10eeb15-4d3d-4ecf-8c1e-df9495831f6c admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:43.282854 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10eeb15-4d3d-4ecf-8c1e-df9495831f6c admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:43.289269 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e137d76-212a-4db3-8f46-8780168285ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:43.289917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e137d76-212a-4db3-8f46-8780168285ff admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:43.290203 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e137d76-212a-4db3-8f46-8780168285ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:43.291202 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e137d76-212a-4db3-8f46-8780168285ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:43.292005 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e137d76-212a-4db3-8f46-8780168285ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:43.293620 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e137d76-212a-4db3-8f46-8780168285ff admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:43.304051 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 982/1969] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:34:43.308586 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af2193-bf27-4d1c-a45b-860c1ac47a09 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:43.318751 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af2193-bf27-4d1c-a45b-860c1ac47a09 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:43.331371 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8d1dc7-e751-41d3-b716-277330610cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:43.331371 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d1dc7-e751-41d3-b716-277330610cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:43.331371 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d1dc7-e751-41d3-b716-277330610cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:43.331371 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d1dc7-e751-41d3-b716-277330610cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:43.331688 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d1dc7-e751-41d3-b716-277330610cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:43.331688 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8d1dc7-e751-41d3-b716-277330610cd0 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:43.343250 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 988/1970] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:34:43.344342 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e137d76-212a-4db3-8f46-8780168285ff admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:43.354627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e137d76-212a-4db3-8f46-8780168285ff admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:43.370985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288d6e5-df9e-44cf-8fb1-095843394826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:43.371933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288d6e5-df9e-44cf-8fb1-095843394826 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:43.372217 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288d6e5-df9e-44cf-8fb1-095843394826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:43.372495 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b288d6e5-df9e-44cf-8fb1-095843394826 admin admin] PATH_INFO: `/v3/projects/70054d80fd034da68fab86c5145ffb1c/users/421aad7e0ebc4d1abac74e91c145545b/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:43.391033 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288d6e5-df9e-44cf-8fb1-095843394826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70054d80fd034da68fab86c5145ffb1c, user_id=421aad7e0ebc4d1abac74e91c145545b, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:43.393745 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b288d6e5-df9e-44cf-8fb1-095843394826 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:43.478156 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 983/1971] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:34:43 2022] PUT /identity/v3/projects/70054d80fd034da68fab86c5145ffb1c/users/421aad7e0ebc4d1abac74e91c145545b/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:43.485114 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684e05f-417e-48e8-9c67-35eb410ea7bd None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:43.485494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684e05f-417e-48e8-9c67-35eb410ea7bd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:43.485870 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684e05f-417e-48e8-9c67-35eb410ea7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:43.504548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4684e05f-417e-48e8-9c67-35eb410ea7bd None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:43.505816 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4684e05f-417e-48e8-9c67-35eb410ea7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:43.570562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 989/1972] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:43.611051 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288d6e5-df9e-44cf-8fb1-095843394826 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:43.613010 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-b288d6e5-df9e-44cf-8fb1-095843394826 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:43.671821 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b288d6e5-df9e-44cf-8fb1-095843394826 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:43.675457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa1e4ba-788d-40ee-8acd-a8f3bda8fd5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:43.678988 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1e4ba-788d-40ee-8acd-a8f3bda8fd5b service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:43.678988 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1e4ba-788d-40ee-8acd-a8f3bda8fd5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:43.678988 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa1e4ba-788d-40ee-8acd-a8f3bda8fd5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:43.688708 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1e4ba-788d-40ee-8acd-a8f3bda8fd5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:43.690147 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa1e4ba-788d-40ee-8acd-a8f3bda8fd5b service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:43.700392 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 984/1973] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:34:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:34:45.013088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684e05f-417e-48e8-9c67-35eb410ea7bd None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:45.118543 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684e05f-417e-48e8-9c67-35eb410ea7bd None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:45.123537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c0288-c178-4bc5-a4dc-dcdb2007b93b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:45.125955 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c0288-c178-4bc5-a4dc-dcdb2007b93b service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:45.126465 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c0288-c178-4bc5-a4dc-dcdb2007b93b service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:45.127089 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96c0288-c178-4bc5-a4dc-dcdb2007b93b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:45.130001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f96c0288-c178-4bc5-a4dc-dcdb2007b93b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:45.220531 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96c0288-c178-4bc5-a4dc-dcdb2007b93b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:45.224705 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96c0288-c178-4bc5-a4dc-dcdb2007b93b service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:45.266938 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 990/1974] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:34:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:34:45.754346 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:45.906845 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a7a72-bda4-4c13-ae5d-97c08ebe7d38 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:45.907710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a7a72-bda4-4c13-ae5d-97c08ebe7d38 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:45.908182 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a7a72-bda4-4c13-ae5d-97c08ebe7d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:45.952715 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-ad2a7a72-bda4-4c13-ae5d-97c08ebe7d38 None None] MFA Rules not processed for user `421aad7e0ebc4d1abac74e91c145545b`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:45.954405 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-ad2a7a72-bda4-4c13-ae5d-97c08ebe7d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:46.071209 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 985/1975] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:46.075631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c0288-c178-4bc5-a4dc-dcdb2007b93b service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:46.092362 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96c0288-c178-4bc5-a4dc-dcdb2007b93b service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:46.100856 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2265a-e79a-4d85-8a28-b53743cb32b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:46.102096 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2265a-e79a-4d85-8a28-b53743cb32b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:46.102382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2265a-e79a-4d85-8a28-b53743cb32b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:46.103063 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee2265a-e79a-4d85-8a28-b53743cb32b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:46.104124 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2265a-e79a-4d85-8a28-b53743cb32b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:46.106087 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee2265a-e79a-4d85-8a28-b53743cb32b6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:46.314178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 991/1976] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:34:46 2022] POST /identity/v3/projects => generated 373 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:34:46.318793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a7a72-bda4-4c13-ae5d-97c08ebe7d38 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:46.334156 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a7a72-bda4-4c13-ae5d-97c08ebe7d38 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:46.402864 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:46.413442 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06431953-dcb5-4a08-ab30-a9183cbc1141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:46.414725 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06431953-dcb5-4a08-ab30-a9183cbc1141 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:46.415010 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06431953-dcb5-4a08-ab30-a9183cbc1141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:46.415308 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06431953-dcb5-4a08-ab30-a9183cbc1141 admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:46.416022 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06431953-dcb5-4a08-ab30-a9183cbc1141 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:46.418024 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06431953-dcb5-4a08-ab30-a9183cbc1141 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:46.534136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 986/1977] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:34:46 2022] POST /identity/v3/users => generated 293 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:34:46.535536 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2265a-e79a-4d85-8a28-b53743cb32b6 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:46.554522 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee2265a-e79a-4d85-8a28-b53743cb32b6 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:46.563284 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b29b31-63ed-42dc-84da-e450a8a269be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:46.563284 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b29b31-63ed-42dc-84da-e450a8a269be admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:46.563284 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b29b31-63ed-42dc-84da-e450a8a269be admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:46.563284 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b29b31-63ed-42dc-84da-e450a8a269be admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:46.563842 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b29b31-63ed-42dc-84da-e450a8a269be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:46.563842 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b29b31-63ed-42dc-84da-e450a8a269be admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:46.574658 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 992/1978] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:34:46.578965 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06431953-dcb5-4a08-ab30-a9183cbc1141 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:46.590045 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06431953-dcb5-4a08-ab30-a9183cbc1141 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:46.595069 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3877fa-5c20-4f93-b077-e5c166f096fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:46.596039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3877fa-5c20-4f93-b077-e5c166f096fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:46.596327 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3877fa-5c20-4f93-b077-e5c166f096fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:46.596610 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3877fa-5c20-4f93-b077-e5c166f096fd admin admin] PATH_INFO: `/v3/projects/b2199044d60d465fa6d564275014ca4c/users/296f7b3dab274b1083efe4105959a15c/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:46.612980 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3877fa-5c20-4f93-b077-e5c166f096fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2199044d60d465fa6d564275014ca4c, user_id=296f7b3dab274b1083efe4105959a15c, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:46.615856 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3877fa-5c20-4f93-b077-e5c166f096fd admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:46.699318 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b29b31-63ed-42dc-84da-e450a8a269be admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:46.700000 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 987/1979] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:34:46 2022] PUT /identity/v3/projects/b2199044d60d465fa6d564275014ca4c/users/296f7b3dab274b1083efe4105959a15c/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:46.711563 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b29b31-63ed-42dc-84da-e450a8a269be admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:46.776885 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80195ea7-3923-4e5d-b563-d488851a63c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:46.777668 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80195ea7-3923-4e5d-b563-d488851a63c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:46.778017 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80195ea7-3923-4e5d-b563-d488851a63c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:46.778360 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80195ea7-3923-4e5d-b563-d488851a63c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:46.779306 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80195ea7-3923-4e5d-b563-d488851a63c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:46.781257 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80195ea7-3923-4e5d-b563-d488851a63c8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:46.791952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 993/1980] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:34:46.797129 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3877fa-5c20-4f93-b077-e5c166f096fd admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:46.811526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3877fa-5c20-4f93-b077-e5c166f096fd admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:46.816357 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67896d8d-da70-4b90-959d-86b04e60d499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:46.817245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67896d8d-da70-4b90-959d-86b04e60d499 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:46.817533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67896d8d-da70-4b90-959d-86b04e60d499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:46.818149 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67896d8d-da70-4b90-959d-86b04e60d499 admin admin] PATH_INFO: `/v3/projects/b2199044d60d465fa6d564275014ca4c/users/296f7b3dab274b1083efe4105959a15c/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:46.819833 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67896d8d-da70-4b90-959d-86b04e60d499 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2199044d60d465fa6d564275014ca4c, user_id=296f7b3dab274b1083efe4105959a15c, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:46.823323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67896d8d-da70-4b90-959d-86b04e60d499 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:46.833928 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 988/1981] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:34:46 2022] PUT /identity/v3/projects/b2199044d60d465fa6d564275014ca4c/users/296f7b3dab274b1083efe4105959a15c/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:47.552342 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80195ea7-3923-4e5d-b563-d488851a63c8 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:47.555464 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-80195ea7-3923-4e5d-b563-d488851a63c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:47.573327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80195ea7-3923-4e5d-b563-d488851a63c8 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:47.595782 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:47.667905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f416b2-e197-4cc9-bb03-bcfed4a79ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:47.669437 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f416b2-e197-4cc9-bb03-bcfed4a79ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:47.670277 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f416b2-e197-4cc9-bb03-bcfed4a79ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:47.670830 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f416b2-e197-4cc9-bb03-bcfed4a79ee3 admin admin] PATH_INFO: `/v3/users/7e2a5236805a4ae9827334ad029c2b8b` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:47.672608 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f416b2-e197-4cc9-bb03-bcfed4a79ee3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e2a5236805a4ae9827334ad029c2b8b)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:47.675250 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f416b2-e197-4cc9-bb03-bcfed4a79ee3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:47.845406 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d1f416b2-e197-4cc9-bb03-bcfed4a79ee3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e2a5236805a4ae9827334ad029c2b8b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:47.846161 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d1f416b2-e197-4cc9-bb03-bcfed4a79ee3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e2a5236805a4ae9827334ad029c2b8b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:47.854720 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d1f416b2-e197-4cc9-bb03-bcfed4a79ee3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e2a5236805a4ae9827334ad029c2b8b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:47.855433 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d1f416b2-e197-4cc9-bb03-bcfed4a79ee3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e2a5236805a4ae9827334ad029c2b8b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:47.932508 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d1f416b2-e197-4cc9-bb03-bcfed4a79ee3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e2a5236805a4ae9827334ad029c2b8b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:47.953960 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 994/1982] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:34:47 2022] DELETE /identity/v3/users/7e2a5236805a4ae9827334ad029c2b8b => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:47.955760 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416313f0-50f7-4db7-a71b-f5c6ddf56fed None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:47.956208 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416313f0-50f7-4db7-a71b-f5c6ddf56fed None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:47.956462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416313f0-50f7-4db7-a71b-f5c6ddf56fed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:47.981065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-416313f0-50f7-4db7-a71b-f5c6ddf56fed None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:47.983094 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-416313f0-50f7-4db7-a71b-f5c6ddf56fed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:48.045413 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 989/1983] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:48.055530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f416b2-e197-4cc9-bb03-bcfed4a79ee3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:48.056987 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d1f416b2-e197-4cc9-bb03-bcfed4a79ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:48.118299 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f416b2-e197-4cc9-bb03-bcfed4a79ee3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:48.122014 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de617006-3b0e-423d-aac8-dc592ce45b0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:48.122669 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de617006-3b0e-423d-aac8-dc592ce45b0b service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:48.122960 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de617006-3b0e-423d-aac8-dc592ce45b0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:48.123241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de617006-3b0e-423d-aac8-dc592ce45b0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:48.134414 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de617006-3b0e-423d-aac8-dc592ce45b0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:48.135954 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de617006-3b0e-423d-aac8-dc592ce45b0b service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:48.145545 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 995/1984] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:34:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:34:48.329274 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb551f-fb30-4c18-8546-b0a5b0b37d16 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:48.329576 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb551f-fb30-4c18-8546-b0a5b0b37d16 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:48.330195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb551f-fb30-4c18-8546-b0a5b0b37d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:48.360456 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-fdcb551f-fb30-4c18-8546-b0a5b0b37d16 None None] MFA Rules not processed for user `296f7b3dab274b1083efe4105959a15c`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:48.364559 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-fdcb551f-fb30-4c18-8546-b0a5b0b37d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:48.459401 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 990/1985] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:48 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:48.468354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de617006-3b0e-423d-aac8-dc592ce45b0b service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:48.469803 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-de617006-3b0e-423d-aac8-dc592ce45b0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:48.518171 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:48.532015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de617006-3b0e-423d-aac8-dc592ce45b0b service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:48.535676 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2820a211-205b-4d39-8998-faa800fb8898 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:48.536330 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2820a211-205b-4d39-8998-faa800fb8898 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:48.536617 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2820a211-205b-4d39-8998-faa800fb8898 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:48.536900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2820a211-205b-4d39-8998-faa800fb8898 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:48.539228 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2820a211-205b-4d39-8998-faa800fb8898 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:48.593790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb551f-fb30-4c18-8546-b0a5b0b37d16 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:48.595924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-fdcb551f-fb30-4c18-8546-b0a5b0b37d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:48.604610 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2820a211-205b-4d39-8998-faa800fb8898 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:48.606148 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2820a211-205b-4d39-8998-faa800fb8898 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:48.612638 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcb551f-fb30-4c18-8546-b0a5b0b37d16 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:48.617862 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87539a43-5728-4ebc-b9c8-d9bf6c22f81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:48.619658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87539a43-5728-4ebc-b9c8-d9bf6c22f81f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:48.619958 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87539a43-5728-4ebc-b9c8-d9bf6c22f81f admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:48.620964 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87539a43-5728-4ebc-b9c8-d9bf6c22f81f admin admin] PATH_INFO: `/v3/projects/a4306927334d4345a57f0fec06893ea6` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:48.622159 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87539a43-5728-4ebc-b9c8-d9bf6c22f81f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4306927334d4345a57f0fec06893ea6)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:48.623421 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 996/1986] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:34:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:34:48.624088 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87539a43-5728-4ebc-b9c8-d9bf6c22f81f admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:48.787286 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:48.802462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-87539a43-5728-4ebc-b9c8-d9bf6c22f81f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4306927334d4345a57f0fec06893ea6', 'request_id': 'req-87539a43-5728-4ebc-b9c8-d9bf6c22f81f'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:48.809547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-87539a43-5728-4ebc-b9c8-d9bf6c22f81f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4306927334d4345a57f0fec06893ea6', 'request_id': 'req-87539a43-5728-4ebc-b9c8-d9bf6c22f81f'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:48.815700 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-87539a43-5728-4ebc-b9c8-d9bf6c22f81f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4306927334d4345a57f0fec06893ea6', 'request_id': 'req-87539a43-5728-4ebc-b9c8-d9bf6c22f81f'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:48.871456 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-87539a43-5728-4ebc-b9c8-d9bf6c22f81f admin admin] The token cache is being invalidate because project a4306927334d4345a57f0fec06893ea6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:34:48.871873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-87539a43-5728-4ebc-b9c8-d9bf6c22f81f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:48.872539 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-87539a43-5728-4ebc-b9c8-d9bf6c22f81f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:48.874289 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 991/1987] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:34:48 2022] DELETE /identity/v3/projects/a4306927334d4345a57f0fec06893ea6 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:48.881391 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde1133-1665-42d2-8715-b8418209cac7 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:48.881678 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde1133-1665-42d2-8715-b8418209cac7 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:48.881953 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbde1133-1665-42d2-8715-b8418209cac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:48.902472 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-fbde1133-1665-42d2-8715-b8418209cac7 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:48.903799 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fbde1133-1665-42d2-8715-b8418209cac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:48.974011 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 997/1988] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:48.982208 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a1082-467c-41a3-8dba-b54a62fa5443 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:48.982758 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a1082-467c-41a3-8dba-b54a62fa5443 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:48.982758 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a1082-467c-41a3-8dba-b54a62fa5443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:49.001217 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-cb3a1082-467c-41a3-8dba-b54a62fa5443 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:49.002479 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-cb3a1082-467c-41a3-8dba-b54a62fa5443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:49.012504 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 992/1989] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:49.016634 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde1133-1665-42d2-8715-b8418209cac7 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:49.027122 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbde1133-1665-42d2-8715-b8418209cac7 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:49.032000 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c642d1c-8c34-4634-9e6c-1d6b2ca57e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:49.032685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c642d1c-8c34-4634-9e6c-1d6b2ca57e8d admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:49.033014 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c642d1c-8c34-4634-9e6c-1d6b2ca57e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:49.033292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c642d1c-8c34-4634-9e6c-1d6b2ca57e8d admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:49.034022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c642d1c-8c34-4634-9e6c-1d6b2ca57e8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:34:49.034330 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c642d1c-8c34-4634-9e6c-1d6b2ca57e8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:49.036058 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c642d1c-8c34-4634-9e6c-1d6b2ca57e8d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:49.048228 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 998/1990] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:34:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:34:49.059029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3a1082-467c-41a3-8dba-b54a62fa5443 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:49.068343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3a1082-467c-41a3-8dba-b54a62fa5443 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:49.073578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b2f01-8213-479d-99ea-44cdbc5a37a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:49.074227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b2f01-8213-479d-99ea-44cdbc5a37a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:49.074510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b2f01-8213-479d-99ea-44cdbc5a37a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:49.074792 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b2f01-8213-479d-99ea-44cdbc5a37a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:49.075531 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b2f01-8213-479d-99ea-44cdbc5a37a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:49.077433 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b2f01-8213-479d-99ea-44cdbc5a37a2 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:49.248871 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 993/1991] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:34:49 2022] POST /identity/v3/projects => generated 377 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:34:49.253396 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c642d1c-8c34-4634-9e6c-1d6b2ca57e8d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:49.264758 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c642d1c-8c34-4634-9e6c-1d6b2ca57e8d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:49.334794 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66faf3-df2a-4fd2-843b-7ad885f42b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:49.335425 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66faf3-df2a-4fd2-843b-7ad885f42b1b admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:49.336418 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66faf3-df2a-4fd2-843b-7ad885f42b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:49.336950 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca66faf3-df2a-4fd2-843b-7ad885f42b1b admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:49.337399 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca66faf3-df2a-4fd2-843b-7ad885f42b1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:49.340052 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca66faf3-df2a-4fd2-843b-7ad885f42b1b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:49.437655 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 999/1992] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:34:49 2022] POST /identity/v3/users => generated 289 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:34:49.441876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b2f01-8213-479d-99ea-44cdbc5a37a2 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:49.453071 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b2f01-8213-479d-99ea-44cdbc5a37a2 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:49.458095 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3b544-31fd-4c41-afe3-c5e7f9fb7bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:49.458363 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3b544-31fd-4c41-afe3-c5e7f9fb7bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:49.458652 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3b544-31fd-4c41-afe3-c5e7f9fb7bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:49.458932 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c3b544-31fd-4c41-afe3-c5e7f9fb7bc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:49.459704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3b544-31fd-4c41-afe3-c5e7f9fb7bc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:49.461302 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c3b544-31fd-4c41-afe3-c5e7f9fb7bc5 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:49.471736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 994/1993] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:34:49.475626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66faf3-df2a-4fd2-843b-7ad885f42b1b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:49.485830 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca66faf3-df2a-4fd2-843b-7ad885f42b1b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:49.490646 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f629c0-1ee1-4c2d-8b6c-1d7420697950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:49.491356 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f629c0-1ee1-4c2d-8b6c-1d7420697950 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:49.492178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f629c0-1ee1-4c2d-8b6c-1d7420697950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:49.492587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f629c0-1ee1-4c2d-8b6c-1d7420697950 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:49.498465 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f629c0-1ee1-4c2d-8b6c-1d7420697950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:49.498465 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f629c0-1ee1-4c2d-8b6c-1d7420697950 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:49.505754 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1000/1994] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:34:49.510279 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3b544-31fd-4c41-afe3-c5e7f9fb7bc5 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:49.520315 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c3b544-31fd-4c41-afe3-c5e7f9fb7bc5 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:49.525502 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61607c-59f3-4f03-8792-5440b0267e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:49.539813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61607c-59f3-4f03-8792-5440b0267e16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:49.539813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61607c-59f3-4f03-8792-5440b0267e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:49.539813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b61607c-59f3-4f03-8792-5440b0267e16 admin admin] PATH_INFO: `/v3/projects/0c97f86db9ec4d01b5a0e2de5978a51c/users/906531c783514dae85ff83c6dbc28f0b/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:49.549902 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b61607c-59f3-4f03-8792-5440b0267e16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c97f86db9ec4d01b5a0e2de5978a51c, user_id=906531c783514dae85ff83c6dbc28f0b, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:49.552708 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b61607c-59f3-4f03-8792-5440b0267e16 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:49.648853 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 995/1995] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:34:49 2022] PUT /identity/v3/projects/0c97f86db9ec4d01b5a0e2de5978a51c/users/906531c783514dae85ff83c6dbc28f0b/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:49.659220 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d695ec7-10a7-43b5-8041-dcf792ca1eaf None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:49.659943 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d695ec7-10a7-43b5-8041-dcf792ca1eaf None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:49.659943 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d695ec7-10a7-43b5-8041-dcf792ca1eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:49.687338 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3d695ec7-10a7-43b5-8041-dcf792ca1eaf None None] MFA Rules not processed for user `906531c783514dae85ff83c6dbc28f0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:49.688557 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3d695ec7-10a7-43b5-8041-dcf792ca1eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:49.751520 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:49.754465 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1001/1996] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:34:49.759508 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61607c-59f3-4f03-8792-5440b0267e16 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:49.820401 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b61607c-59f3-4f03-8792-5440b0267e16 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:49.824924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58828f69-9597-4afa-898a-bd587d0d5e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:49.825582 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58828f69-9597-4afa-898a-bd587d0d5e9a admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:49.825867 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58828f69-9597-4afa-898a-bd587d0d5e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:49.826139 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58828f69-9597-4afa-898a-bd587d0d5e9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:49.826886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58828f69-9597-4afa-898a-bd587d0d5e9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:49.828822 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58828f69-9597-4afa-898a-bd587d0d5e9a admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:49.972519 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 996/1997] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:34:49 2022] POST /identity/v3/projects => generated 375 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:34:49.974399 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d695ec7-10a7-43b5-8041-dcf792ca1eaf None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:49.986547 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d695ec7-10a7-43b5-8041-dcf792ca1eaf None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:50.045270 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578498f-0e48-4a31-96d4-5e5d66c4679d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:50.045985 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578498f-0e48-4a31-96d4-5e5d66c4679d admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:50.046271 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578498f-0e48-4a31-96d4-5e5d66c4679d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:50.046549 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e578498f-0e48-4a31-96d4-5e5d66c4679d admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:50.047244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e578498f-0e48-4a31-96d4-5e5d66c4679d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:50.049219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e578498f-0e48-4a31-96d4-5e5d66c4679d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:50.185197 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1002/1998] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:34:49 2022] POST /identity/v3/users => generated 288 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:34:50.189286 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58828f69-9597-4afa-898a-bd587d0d5e9a admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:50.199634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58828f69-9597-4afa-898a-bd587d0d5e9a admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:50.204132 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d0591-ccf0-41ee-ba46-e760bfc2f205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:50.204793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d0591-ccf0-41ee-ba46-e760bfc2f205 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:50.205078 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d0591-ccf0-41ee-ba46-e760bfc2f205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:50.205356 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d0591-ccf0-41ee-ba46-e760bfc2f205 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:50.206084 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8d0591-ccf0-41ee-ba46-e760bfc2f205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:50.207718 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8d0591-ccf0-41ee-ba46-e760bfc2f205 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:50.218932 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 997/1999] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:34:50.223041 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578498f-0e48-4a31-96d4-5e5d66c4679d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:50.233895 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e578498f-0e48-4a31-96d4-5e5d66c4679d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:50.238497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d249ba0-d345-4af3-bfd5-81e268f61350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:50.239153 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d249ba0-d345-4af3-bfd5-81e268f61350 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:50.239469 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d249ba0-d345-4af3-bfd5-81e268f61350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:50.239749 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d249ba0-d345-4af3-bfd5-81e268f61350 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:50.240507 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d249ba0-d345-4af3-bfd5-81e268f61350 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:50.242255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d249ba0-d345-4af3-bfd5-81e268f61350 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:50.256727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1003/2000] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:34:50.257963 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d0591-ccf0-41ee-ba46-e760bfc2f205 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:50.268037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d0591-ccf0-41ee-ba46-e760bfc2f205 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:50.273901 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc261ba-55fa-4de2-a0ac-98969794fddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:50.274961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc261ba-55fa-4de2-a0ac-98969794fddc admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:50.275364 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc261ba-55fa-4de2-a0ac-98969794fddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:50.275646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc261ba-55fa-4de2-a0ac-98969794fddc admin admin] PATH_INFO: `/v3/projects/c85724bcfa2644b78244b417d8dac24f/users/0133caf8926746e48ed053076551421e/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:50.299625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc261ba-55fa-4de2-a0ac-98969794fddc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c85724bcfa2644b78244b417d8dac24f, user_id=0133caf8926746e48ed053076551421e, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:50.302280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc261ba-55fa-4de2-a0ac-98969794fddc admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:50.343007 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d249ba0-d345-4af3-bfd5-81e268f61350 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:50.344856 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6d249ba0-d345-4af3-bfd5-81e268f61350 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:50.392364 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:50.397376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 998/2001] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:34:50 2022] PUT /identity/v3/projects/c85724bcfa2644b78244b417d8dac24f/users/0133caf8926746e48ed053076551421e/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:50.404178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d249ba0-d345-4af3-bfd5-81e268f61350 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:50.409449 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f7f80-ee55-4c0b-98d6-d954500ca942 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:50.409751 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f7f80-ee55-4c0b-98d6-d954500ca942 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:50.410031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483f7f80-ee55-4c0b-98d6-d954500ca942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:50.425369 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:50.436997 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-483f7f80-ee55-4c0b-98d6-d954500ca942 None None] MFA Rules not processed for user `0133caf8926746e48ed053076551421e`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:50.438186 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-483f7f80-ee55-4c0b-98d6-d954500ca942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:50.456418 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf65fb4-916f-40e7-8d26-8e8d53ffa886 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:50.457114 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf65fb4-916f-40e7-8d26-8e8d53ffa886 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:50.457406 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf65fb4-916f-40e7-8d26-8e8d53ffa886 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:50.457688 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf65fb4-916f-40e7-8d26-8e8d53ffa886 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:50.459615 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7bf65fb4-916f-40e7-8d26-8e8d53ffa886 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:50.511470 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 999/2002] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:34:50.516258 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f7f80-ee55-4c0b-98d6-d954500ca942 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:50.536160 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf65fb4-916f-40e7-8d26-8e8d53ffa886 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:50.537703 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf65fb4-916f-40e7-8d26-8e8d53ffa886 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:50.547681 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1004/2003] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:34:50 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:34:50.582586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483f7f80-ee55-4c0b-98d6-d954500ca942 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:50.588387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f770e-afeb-45a1-ac13-52be22877074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:50.589054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f770e-afeb-45a1-ac13-52be22877074 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:50.589342 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f770e-afeb-45a1-ac13-52be22877074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:50.589622 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f770e-afeb-45a1-ac13-52be22877074 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:50.590313 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f770e-afeb-45a1-ac13-52be22877074 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:50.592296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f770e-afeb-45a1-ac13-52be22877074 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:50.666645 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf65fb4-916f-40e7-8d26-8e8d53ffa886 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:50.668643 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7bf65fb4-916f-40e7-8d26-8e8d53ffa886 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:50.675286 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1000/2004] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:34:50 2022] POST /identity/v3/projects => generated 375 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:34:50.682563 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf65fb4-916f-40e7-8d26-8e8d53ffa886 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:50.683585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f770e-afeb-45a1-ac13-52be22877074 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:50.752917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f770e-afeb-45a1-ac13-52be22877074 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:50.758285 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42c918-2c4d-426f-b4d6-05d21a2cbe2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:50.758446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8270555-e0a9-4917-ba5b-efb41baa8ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:50.758973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42c918-2c4d-426f-b4d6-05d21a2cbe2d admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:50.759171 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8270555-e0a9-4917-ba5b-efb41baa8ce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:50.759306 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42c918-2c4d-426f-b4d6-05d21a2cbe2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:50.759490 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8270555-e0a9-4917-ba5b-efb41baa8ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:50.759583 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42c918-2c4d-426f-b4d6-05d21a2cbe2d admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:50.759778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8270555-e0a9-4917-ba5b-efb41baa8ce7 admin admin] PATH_INFO: `/v3/users/3571d5ce8fbb4421b8af34b8a43d6fe4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:50.760249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b42c918-2c4d-426f-b4d6-05d21a2cbe2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:50.761043 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8270555-e0a9-4917-ba5b-efb41baa8ce7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3571d5ce8fbb4421b8af34b8a43d6fe4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:50.762227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b42c918-2c4d-426f-b4d6-05d21a2cbe2d admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:50.762984 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8270555-e0a9-4917-ba5b-efb41baa8ce7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:50.848056 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1001/2005] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:34:50 2022] POST /identity/v3/users => generated 294 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:34:50.852131 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42c918-2c4d-426f-b4d6-05d21a2cbe2d admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:50.858441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:50.861965 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42c918-2c4d-426f-b4d6-05d21a2cbe2d admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:50.867164 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920c0d3-a9f3-459d-88b2-a95e779467bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:50.867846 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920c0d3-a9f3-459d-88b2-a95e779467bb admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:50.868141 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920c0d3-a9f3-459d-88b2-a95e779467bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:50.868425 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2920c0d3-a9f3-459d-88b2-a95e779467bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:50.869153 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920c0d3-a9f3-459d-88b2-a95e779467bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:50.870760 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2920c0d3-a9f3-459d-88b2-a95e779467bb admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:50.885441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1002/2006] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:34:50.886444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920c0d3-a9f3-459d-88b2-a95e779467bb admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:50.896320 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2920c0d3-a9f3-459d-88b2-a95e779467bb admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:50.901629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff9b59-e079-4312-b32e-36317dbb75ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:50.902508 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff9b59-e079-4312-b32e-36317dbb75ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:50.907691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff9b59-e079-4312-b32e-36317dbb75ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:50.907691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ff9b59-e079-4312-b32e-36317dbb75ca admin admin] PATH_INFO: `/v3/projects/21d082d5c47446abb54ca92d1ab88421/users/4c06d450248b4884a51bddddc9e1c6f3/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:50.920110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff9b59-e079-4312-b32e-36317dbb75ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=21d082d5c47446abb54ca92d1ab88421, user_id=4c06d450248b4884a51bddddc9e1c6f3, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:50.922869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ff9b59-e079-4312-b32e-36317dbb75ca admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:50.988903 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c8270555-e0a9-4917-ba5b-efb41baa8ce7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3571d5ce8fbb4421b8af34b8a43d6fe4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:50.989719 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c8270555-e0a9-4917-ba5b-efb41baa8ce7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3571d5ce8fbb4421b8af34b8a43d6fe4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:50.998926 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1003/2007] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:34:50 2022] PUT /identity/v3/projects/21d082d5c47446abb54ca92d1ab88421/users/4c06d450248b4884a51bddddc9e1c6f3/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:34:51.000087 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c8270555-e0a9-4917-ba5b-efb41baa8ce7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3571d5ce8fbb4421b8af34b8a43d6fe4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:51.003879 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c8270555-e0a9-4917-ba5b-efb41baa8ce7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3571d5ce8fbb4421b8af34b8a43d6fe4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:51.003983 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff9b59-e079-4312-b32e-36317dbb75ca admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:51.004482 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d8ff9b59-e079-4312-b32e-36317dbb75ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:51.064252 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c8270555-e0a9-4917-ba5b-efb41baa8ce7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3571d5ce8fbb4421b8af34b8a43d6fe4'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:51.078562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:51.091064 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1005/2008] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:34:50 2022] DELETE /identity/v3/users/3571d5ce8fbb4421b8af34b8a43d6fe4 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:34:51.095597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185996e4-3515-4469-b755-2273276d6e51 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:51.095973 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185996e4-3515-4469-b755-2273276d6e51 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:51.096258 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185996e4-3515-4469-b755-2273276d6e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:51.116018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-185996e4-3515-4469-b755-2273276d6e51 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:51.117889 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-185996e4-3515-4469-b755-2273276d6e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:51.120753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ff9b59-e079-4312-b32e-36317dbb75ca admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:51.186577 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2392038-be6f-490a-ac73-a6e2a04e76ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:51.187307 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2392038-be6f-490a-ac73-a6e2a04e76ae admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:51.187590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2392038-be6f-490a-ac73-a6e2a04e76ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:51.187921 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2392038-be6f-490a-ac73-a6e2a04e76ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:51.188598 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2392038-be6f-490a-ac73-a6e2a04e76ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:51.194037 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1006/2009] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:51.194179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2392038-be6f-490a-ac73-a6e2a04e76ae admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:51.203484 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1004/2010] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:51 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:34:51.203943 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185996e4-3515-4469-b755-2273276d6e51 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:51.205772 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-185996e4-3515-4469-b755-2273276d6e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:51.207092 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2392038-be6f-490a-ac73-a6e2a04e76ae admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:51.217925 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2392038-be6f-490a-ac73-a6e2a04e76ae admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:51.222886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17ca985-0224-4f0b-a69f-766cbf050368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:51.223825 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ca985-0224-4f0b-a69f-766cbf050368 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:51.224109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ca985-0224-4f0b-a69f-766cbf050368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:51.227717 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17ca985-0224-4f0b-a69f-766cbf050368 admin admin] PATH_INFO: `/v3/projects/21d082d5c47446abb54ca92d1ab88421/users/4c06d450248b4884a51bddddc9e1c6f3/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:51.227717 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ca985-0224-4f0b-a69f-766cbf050368 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21d082d5c47446abb54ca92d1ab88421, user_id=4c06d450248b4884a51bddddc9e1c6f3, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:51.229588 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17ca985-0224-4f0b-a69f-766cbf050368 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:51.240643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1005/2011] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:34:51 2022] PUT /identity/v3/projects/21d082d5c47446abb54ca92d1ab88421/users/4c06d450248b4884a51bddddc9e1c6f3/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:51.251211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ea2cf-ac35-4e4d-b60c-e7d2e42c2f47 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:51.251532 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ea2cf-ac35-4e4d-b60c-e7d2e42c2f47 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:51.251812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ea2cf-ac35-4e4d-b60c-e7d2e42c2f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:51.268320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185996e4-3515-4469-b755-2273276d6e51 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:51.278550 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-479ea2cf-ac35-4e4d-b60c-e7d2e42c2f47 None None] MFA Rules not processed for user `4c06d450248b4884a51bddddc9e1c6f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:51.280101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-479ea2cf-ac35-4e4d-b60c-e7d2e42c2f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:51.330076 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3d8ef-6afd-42ee-8b59-c4b79c8afd03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:51.330806 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3d8ef-6afd-42ee-8b59-c4b79c8afd03 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:51.331157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3d8ef-6afd-42ee-8b59-c4b79c8afd03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:51.331548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3d8ef-6afd-42ee-8b59-c4b79c8afd03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:51.341529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:51.367360 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1006/2012] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:51 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:34:51.381942 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ea2cf-ac35-4e4d-b60c-e7d2e42c2f47 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:51.383552 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-479ea2cf-ac35-4e4d-b60c-e7d2e42c2f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:51.398109 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd3d8ef-6afd-42ee-8b59-c4b79c8afd03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:51.399549 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd3d8ef-6afd-42ee-8b59-c4b79c8afd03 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:51.410394 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1007/2013] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:34:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:34:51.444557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479ea2cf-ac35-4e4d-b60c-e7d2e42c2f47 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:51.448445 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57211a4-96b4-40bf-92d2-a4a846fe6094 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:51.449098 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57211a4-96b4-40bf-92d2-a4a846fe6094 service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:51.449574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57211a4-96b4-40bf-92d2-a4a846fe6094 service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:51.449702 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57211a4-96b4-40bf-92d2-a4a846fe6094 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:51.451730 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-f57211a4-96b4-40bf-92d2-a4a846fe6094 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:51.524477 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57211a4-96b4-40bf-92d2-a4a846fe6094 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:51.526042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57211a4-96b4-40bf-92d2-a4a846fe6094 service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:51.535572 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1007/2014] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:34:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:34:51.642660 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:51.896612 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3d8ef-6afd-42ee-8b59-c4b79c8afd03 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:51.897223 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9fd3d8ef-6afd-42ee-8b59-c4b79c8afd03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:51.916754 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3d8ef-6afd-42ee-8b59-c4b79c8afd03 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:51.921424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2d744-cd5a-4de5-beb3-fd1eabe037df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:51.922297 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2d744-cd5a-4de5-beb3-fd1eabe037df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:51.922583 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2d744-cd5a-4de5-beb3-fd1eabe037df admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:51.922861 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2d744-cd5a-4de5-beb3-fd1eabe037df admin admin] PATH_INFO: `/v3/projects/78f728f0f9af407183ba0b45d9689d0f` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:51.924109 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2d744-cd5a-4de5-beb3-fd1eabe037df admin admin] RBAC: Authorizing `identity:delete_project(project_id=78f728f0f9af407183ba0b45d9689d0f)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:51.926070 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2d744-cd5a-4de5-beb3-fd1eabe037df admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:52.084407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ccb2d744-cd5a-4de5-beb3-fd1eabe037df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78f728f0f9af407183ba0b45d9689d0f', 'request_id': 'req-ccb2d744-cd5a-4de5-beb3-fd1eabe037df'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:52.226568 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ccb2d744-cd5a-4de5-beb3-fd1eabe037df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78f728f0f9af407183ba0b45d9689d0f', 'request_id': 'req-ccb2d744-cd5a-4de5-beb3-fd1eabe037df'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:52.233444 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ccb2d744-cd5a-4de5-beb3-fd1eabe037df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78f728f0f9af407183ba0b45d9689d0f', 'request_id': 'req-ccb2d744-cd5a-4de5-beb3-fd1eabe037df'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:52.246355 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ccb2d744-cd5a-4de5-beb3-fd1eabe037df admin admin] The token cache is being invalidate because project 78f728f0f9af407183ba0b45d9689d0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:34:52.246992 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ccb2d744-cd5a-4de5-beb3-fd1eabe037df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:52.247968 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ccb2d744-cd5a-4de5-beb3-fd1eabe037df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:52.249625 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1008/2015] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:34:51 2022] DELETE /identity/v3/projects/78f728f0f9af407183ba0b45d9689d0f => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:52.254848 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57211a4-96b4-40bf-92d2-a4a846fe6094 service cinder] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:52.272417 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57211a4-96b4-40bf-92d2-a4a846fe6094 service cinder] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:52.354873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f720b-be3c-4ed3-8bbc-6270adcada69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:52.355810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f720b-be3c-4ed3-8bbc-6270adcada69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:52.356089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f720b-be3c-4ed3-8bbc-6270adcada69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:52.356372 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79f720b-be3c-4ed3-8bbc-6270adcada69 admin admin] PATH_INFO: `/v3/users/ac316d9eb0194942aed41a34a2f1c503` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:52.357583 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f720b-be3c-4ed3-8bbc-6270adcada69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac316d9eb0194942aed41a34a2f1c503)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:52.359795 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f79f720b-be3c-4ed3-8bbc-6270adcada69 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:52.571942 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:52.576644 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-f79f720b-be3c-4ed3-8bbc-6270adcada69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac316d9eb0194942aed41a34a2f1c503'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:52.699114 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-f79f720b-be3c-4ed3-8bbc-6270adcada69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac316d9eb0194942aed41a34a2f1c503'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:52.712473 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-f79f720b-be3c-4ed3-8bbc-6270adcada69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac316d9eb0194942aed41a34a2f1c503'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:52.713225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-f79f720b-be3c-4ed3-8bbc-6270adcada69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac316d9eb0194942aed41a34a2f1c503'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:52.713840 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-f79f720b-be3c-4ed3-8bbc-6270adcada69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac316d9eb0194942aed41a34a2f1c503'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:52.726965 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1008/2016] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:34:52 2022] DELETE /identity/v3/users/ac316d9eb0194942aed41a34a2f1c503 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:53.291227 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2d744-cd5a-4de5-beb3-fd1eabe037df admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:53.292784 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ccb2d744-cd5a-4de5-beb3-fd1eabe037df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:53.316644 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2d744-cd5a-4de5-beb3-fd1eabe037df admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:53.392447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc53f0-d3be-49e1-b787-c9310c2ce29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:53.393342 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc53f0-d3be-49e1-b787-c9310c2ce29c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:53.393342 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc53f0-d3be-49e1-b787-c9310c2ce29c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:53.393634 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc53f0-d3be-49e1-b787-c9310c2ce29c admin admin] PATH_INFO: `/v3/projects/ebbe2271c89e4ccabd92b351d0a2ce4c` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:53.394929 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc53f0-d3be-49e1-b787-c9310c2ce29c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebbe2271c89e4ccabd92b351d0a2ce4c)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:53.396877 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50bc53f0-d3be-49e1-b787-c9310c2ce29c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:53.558717 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f720b-be3c-4ed3-8bbc-6270adcada69 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:53.560610 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-f79f720b-be3c-4ed3-8bbc-6270adcada69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:53.578178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-50bc53f0-d3be-49e1-b787-c9310c2ce29c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebbe2271c89e4ccabd92b351d0a2ce4c', 'request_id': 'req-50bc53f0-d3be-49e1-b787-c9310c2ce29c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:53.626668 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79f720b-be3c-4ed3-8bbc-6270adcada69 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:53.676066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a539b7b-04b2-4167-a1f4-b4538fc1b7ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:53.676788 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a539b7b-04b2-4167-a1f4-b4538fc1b7ed service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:53.677080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a539b7b-04b2-4167-a1f4-b4538fc1b7ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:53.677359 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a539b7b-04b2-4167-a1f4-b4538fc1b7ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:53.679449 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-9a539b7b-04b2-4167-a1f4-b4538fc1b7ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:53.684500 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-50bc53f0-d3be-49e1-b787-c9310c2ce29c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebbe2271c89e4ccabd92b351d0a2ce4c', 'request_id': 'req-50bc53f0-d3be-49e1-b787-c9310c2ce29c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:53.692619 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-50bc53f0-d3be-49e1-b787-c9310c2ce29c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebbe2271c89e4ccabd92b351d0a2ce4c', 'request_id': 'req-50bc53f0-d3be-49e1-b787-c9310c2ce29c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:53.709331 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-50bc53f0-d3be-49e1-b787-c9310c2ce29c admin admin] The token cache is being invalidate because project ebbe2271c89e4ccabd92b351d0a2ce4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:34:53.709728 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-50bc53f0-d3be-49e1-b787-c9310c2ce29c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:53.715969 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-50bc53f0-d3be-49e1-b787-c9310c2ce29c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:53.715969 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1009/2017] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:34:53 2022] DELETE /identity/v3/projects/ebbe2271c89e4ccabd92b351d0a2ce4c => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:34:53.717312 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc53f0-d3be-49e1-b787-c9310c2ce29c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:53.718759 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-50bc53f0-d3be-49e1-b787-c9310c2ce29c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:53.735002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:53.741937 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a539b7b-04b2-4167-a1f4-b4538fc1b7ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:53.744073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a539b7b-04b2-4167-a1f4-b4538fc1b7ed service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:53.753800 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1009/2018] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:34:53 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:34:53.790959 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc53f0-d3be-49e1-b787-c9310c2ce29c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:53.796511 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae913d17-5c87-4f66-b4e7-a0c859addba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:53.797447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae913d17-5c87-4f66-b4e7-a0c859addba4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:53.797766 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae913d17-5c87-4f66-b4e7-a0c859addba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:53.798046 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae913d17-5c87-4f66-b4e7-a0c859addba4 admin admin] PATH_INFO: `/v3/users/46eff453481747aa91323d3c3beda651` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:53.799373 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae913d17-5c87-4f66-b4e7-a0c859addba4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46eff453481747aa91323d3c3beda651)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:53.801319 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae913d17-5c87-4f66-b4e7-a0c859addba4 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:54.147253 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ae913d17-5c87-4f66-b4e7-a0c859addba4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46eff453481747aa91323d3c3beda651'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:54.147935 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ae913d17-5c87-4f66-b4e7-a0c859addba4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46eff453481747aa91323d3c3beda651'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:54.157475 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ae913d17-5c87-4f66-b4e7-a0c859addba4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46eff453481747aa91323d3c3beda651'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:54.158152 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ae913d17-5c87-4f66-b4e7-a0c859addba4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46eff453481747aa91323d3c3beda651'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:54.299163 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ae913d17-5c87-4f66-b4e7-a0c859addba4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46eff453481747aa91323d3c3beda651'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:54.317879 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1010/2019] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:34:53 2022] DELETE /identity/v3/users/46eff453481747aa91323d3c3beda651 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:34:55.045247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a539b7b-04b2-4167-a1f4-b4538fc1b7ed service cinder] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:55.047728 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-9a539b7b-04b2-4167-a1f4-b4538fc1b7ed service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:55.066088 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a539b7b-04b2-4167-a1f4-b4538fc1b7ed service cinder] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:55.101206 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:55.136267 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2b8fd0-9d64-47d6-8af3-5a1f57c20fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:55.137982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b8fd0-9d64-47d6-8af3-5a1f57c20fd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:55.137982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b8fd0-9d64-47d6-8af3-5a1f57c20fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:55.137982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2b8fd0-9d64-47d6-8af3-5a1f57c20fd3 admin admin] PATH_INFO: `/v3/projects/8bae28169f164612835d8123bdeb2744` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:55.139702 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b8fd0-9d64-47d6-8af3-5a1f57c20fd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bae28169f164612835d8123bdeb2744)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:55.141031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2b8fd0-9d64-47d6-8af3-5a1f57c20fd3 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:55.231570 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1b2b8fd0-9d64-47d6-8af3-5a1f57c20fd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bae28169f164612835d8123bdeb2744', 'request_id': 'req-1b2b8fd0-9d64-47d6-8af3-5a1f57c20fd3'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:55.241956 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1b2b8fd0-9d64-47d6-8af3-5a1f57c20fd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bae28169f164612835d8123bdeb2744', 'request_id': 'req-1b2b8fd0-9d64-47d6-8af3-5a1f57c20fd3'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:55.252046 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1b2b8fd0-9d64-47d6-8af3-5a1f57c20fd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bae28169f164612835d8123bdeb2744', 'request_id': 'req-1b2b8fd0-9d64-47d6-8af3-5a1f57c20fd3'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:55.332742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1b2b8fd0-9d64-47d6-8af3-5a1f57c20fd3 admin admin] The token cache is being invalidate because project 8bae28169f164612835d8123bdeb2744 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:34:55.332959 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1b2b8fd0-9d64-47d6-8af3-5a1f57c20fd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:55.333617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1b2b8fd0-9d64-47d6-8af3-5a1f57c20fd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:34:55.335179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1010/2020] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:34:55 2022] DELETE /identity/v3/projects/8bae28169f164612835d8123bdeb2744 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:34:55.340263 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ebc8ee-1f6d-49ab-a13b-8881852068e8 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:55.340733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ebc8ee-1f6d-49ab-a13b-8881852068e8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:55.340843 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ebc8ee-1f6d-49ab-a13b-8881852068e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:55.358847 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d9ebc8ee-1f6d-49ab-a13b-8881852068e8 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:55.360086 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d9ebc8ee-1f6d-49ab-a13b-8881852068e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:55.418272 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1011/2021] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:55.426017 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951932a7-a815-466d-b8d0-978e66af18ce None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:55.426311 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951932a7-a815-466d-b8d0-978e66af18ce None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:55.426616 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951932a7-a815-466d-b8d0-978e66af18ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:55.445945 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-951932a7-a815-466d-b8d0-978e66af18ce None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:55.447717 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-951932a7-a815-466d-b8d0-978e66af18ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:55.455878 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1011/2022] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:55.460001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ebc8ee-1f6d-49ab-a13b-8881852068e8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:55.470563 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ebc8ee-1f6d-49ab-a13b-8881852068e8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:55.475171 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae61262-027d-427b-a40a-eac7952a6899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:55.475922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae61262-027d-427b-a40a-eac7952a6899 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:55.476208 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae61262-027d-427b-a40a-eac7952a6899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:55.476513 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae61262-027d-427b-a40a-eac7952a6899 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:55.477645 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae61262-027d-427b-a40a-eac7952a6899 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:34:55.477956 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae61262-027d-427b-a40a-eac7952a6899 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:55.479669 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae61262-027d-427b-a40a-eac7952a6899 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:55.493337 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1012/2023] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:34:55.497121 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951932a7-a815-466d-b8d0-978e66af18ce None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:55.506559 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951932a7-a815-466d-b8d0-978e66af18ce None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:55.511972 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edb467-7e07-4360-9ddb-7ad29dbb2785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:55.516810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edb467-7e07-4360-9ddb-7ad29dbb2785 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:55.516810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edb467-7e07-4360-9ddb-7ad29dbb2785 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:55.516810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edb467-7e07-4360-9ddb-7ad29dbb2785 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:55.516810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51edb467-7e07-4360-9ddb-7ad29dbb2785 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:55.516810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51edb467-7e07-4360-9ddb-7ad29dbb2785 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:55.626372 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1012/2024] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:34:55 2022] POST /identity/v3/projects => generated 369 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:34:55.630419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae61262-027d-427b-a40a-eac7952a6899 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:55.641516 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae61262-027d-427b-a40a-eac7952a6899 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:55.658859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:55.701781 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c9348-6f84-4f8e-b3a3-12f2cf70d81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:55.702458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c9348-6f84-4f8e-b3a3-12f2cf70d81b admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:55.702743 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c9348-6f84-4f8e-b3a3-12f2cf70d81b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:55.703026 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c9348-6f84-4f8e-b3a3-12f2cf70d81b admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:55.703792 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c9348-6f84-4f8e-b3a3-12f2cf70d81b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:55.706633 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c9348-6f84-4f8e-b3a3-12f2cf70d81b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:55.786603 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1013/2025] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:34:55 2022] POST /identity/v3/users => generated 285 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:34:55.793844 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edb467-7e07-4360-9ddb-7ad29dbb2785 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:55.804477 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edb467-7e07-4360-9ddb-7ad29dbb2785 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:55.809371 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbef69a-b95a-4b3f-9d20-f2e08e7fc2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:55.810034 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbef69a-b95a-4b3f-9d20-f2e08e7fc2f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:55.810325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbef69a-b95a-4b3f-9d20-f2e08e7fc2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:55.810607 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbef69a-b95a-4b3f-9d20-f2e08e7fc2f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:55.811400 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbef69a-b95a-4b3f-9d20-f2e08e7fc2f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:55.813042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbef69a-b95a-4b3f-9d20-f2e08e7fc2f9 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:55.824081 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1013/2026] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:34:55.827985 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c9348-6f84-4f8e-b3a3-12f2cf70d81b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:55.838658 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c9348-6f84-4f8e-b3a3-12f2cf70d81b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:55.843530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98dacd49-0670-4358-a62d-e5279d4bd37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:55.844130 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dacd49-0670-4358-a62d-e5279d4bd37e admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:55.844410 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dacd49-0670-4358-a62d-e5279d4bd37e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:55.844683 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98dacd49-0670-4358-a62d-e5279d4bd37e admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:55.845482 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dacd49-0670-4358-a62d-e5279d4bd37e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:55.847096 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98dacd49-0670-4358-a62d-e5279d4bd37e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:55.858338 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1014/2027] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:34:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:34:55.862442 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbef69a-b95a-4b3f-9d20-f2e08e7fc2f9 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:55.872813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbef69a-b95a-4b3f-9d20-f2e08e7fc2f9 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:55.877610 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e676bcf-5689-4391-9513-b91df5362608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:55.878504 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e676bcf-5689-4391-9513-b91df5362608 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:55.878785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e676bcf-5689-4391-9513-b91df5362608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:55.879056 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e676bcf-5689-4391-9513-b91df5362608 admin admin] PATH_INFO: `/v3/projects/c6b045f359b149c4aab039644ee02a61/users/98fd19fec73e4a368b00049588dcc4a6/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:55.894693 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e676bcf-5689-4391-9513-b91df5362608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6b045f359b149c4aab039644ee02a61, user_id=98fd19fec73e4a368b00049588dcc4a6, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:55.897432 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e676bcf-5689-4391-9513-b91df5362608 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:55.961899 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1014/2028] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:34:55 2022] PUT /identity/v3/projects/c6b045f359b149c4aab039644ee02a61/users/98fd19fec73e4a368b00049588dcc4a6/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:34:55.972781 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d2bc7-98f9-494f-a37b-2de087936c1a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:55.973071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d2bc7-98f9-494f-a37b-2de087936c1a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:55.973353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d2bc7-98f9-494f-a37b-2de087936c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:56.004150 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6c5d2bc7-98f9-494f-a37b-2de087936c1a None None] MFA Rules not processed for user `98fd19fec73e4a368b00049588dcc4a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:34:56.005369 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6c5d2bc7-98f9-494f-a37b-2de087936c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:56.073668 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1015/2029] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:34:55 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:34:56.084979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e676bcf-5689-4391-9513-b91df5362608 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:56.087928 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-1e676bcf-5689-4391-9513-b91df5362608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:56.144730 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e676bcf-5689-4391-9513-b91df5362608 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:56.148322 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef69c3-fa87-4f3d-bb30-e2524f624b32 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7a61d85908d42698b27b9fc00daa832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:56.149651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef69c3-fa87-4f3d-bb30-e2524f624b32 service glance] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:56.150006 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef69c3-fa87-4f3d-bb30-e2524f624b32 service glance] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:56.150287 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef69c3-fa87-4f3d-bb30-e2524f624b32 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:56.158654 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef69c3-fa87-4f3d-bb30-e2524f624b32 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:56.160106 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef69c3-fa87-4f3d-bb30-e2524f624b32 service glance] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:56.163547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:56.170003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1015/2030] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:34:56 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:34:57.661908 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d2bc7-98f9-494f-a37b-2de087936c1a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:34:57.663395 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6c5d2bc7-98f9-494f-a37b-2de087936c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:57.722741 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d2bc7-98f9-494f-a37b-2de087936c1a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:34:57.726636 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad2d8b-2d62-4976-a725-3fe6a652ead1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:34:57.727258 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad2d8b-2d62-4976-a725-3fe6a652ead1 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:34:57.727575 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad2d8b-2d62-4976-a725-3fe6a652ead1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:34:57.727860 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad2d8b-2d62-4976-a725-3fe6a652ead1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:34:57.729873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8fad2d8b-2d62-4976-a725-3fe6a652ead1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:34:57.781200 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:34:57.812881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad2d8b-2d62-4976-a725-3fe6a652ead1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:34:57.814781 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad2d8b-2d62-4976-a725-3fe6a652ead1 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:34:57.828367 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1016/2031] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:34:57 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:35:02.721343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef69c3-fa87-4f3d-bb30-e2524f624b32 service glance] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:02.723612 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-03ef69c3-fa87-4f3d-bb30-e2524f624b32 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:02.812491 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef69c3-fa87-4f3d-bb30-e2524f624b32 service glance] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:02.817282 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f3bef-7091-4b95-86a1-3ab2f3ae440f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:02.818400 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f3bef-7091-4b95-86a1-3ab2f3ae440f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:02.818847 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f3bef-7091-4b95-86a1-3ab2f3ae440f admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:02.819325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f3bef-7091-4b95-86a1-3ab2f3ae440f admin admin] PATH_INFO: `/v3/users/906531c783514dae85ff83c6dbc28f0b` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:02.820857 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f3bef-7091-4b95-86a1-3ab2f3ae440f admin admin] RBAC: Authorizing `identity:delete_user(user_id=906531c783514dae85ff83c6dbc28f0b)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:02.823696 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5f3bef-7091-4b95-86a1-3ab2f3ae440f admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:03.021607 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1e5f3bef-7091-4b95-86a1-3ab2f3ae440f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '906531c783514dae85ff83c6dbc28f0b'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:03.150966 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1e5f3bef-7091-4b95-86a1-3ab2f3ae440f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '906531c783514dae85ff83c6dbc28f0b'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:03.166822 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1e5f3bef-7091-4b95-86a1-3ab2f3ae440f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '906531c783514dae85ff83c6dbc28f0b'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:03.167549 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1e5f3bef-7091-4b95-86a1-3ab2f3ae440f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '906531c783514dae85ff83c6dbc28f0b'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:03.168158 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1e5f3bef-7091-4b95-86a1-3ab2f3ae440f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '906531c783514dae85ff83c6dbc28f0b'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:03.183101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1016/2032] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:35:02 2022] DELETE /identity/v3/users/906531c783514dae85ff83c6dbc28f0b => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:03.188761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507740b4-0544-4ff7-847a-158969db206b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:03.188761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507740b4-0544-4ff7-847a-158969db206b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:03.188761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507740b4-0544-4ff7-847a-158969db206b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:03.211253 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-507740b4-0544-4ff7-847a-158969db206b None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:03.217479 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-507740b4-0544-4ff7-847a-158969db206b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:03.274824 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1017/2033] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:03.284420 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f3bef-7091-4b95-86a1-3ab2f3ae440f admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:03.285922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-1e5f3bef-7091-4b95-86a1-3ab2f3ae440f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:03.328479 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507740b4-0544-4ff7-847a-158969db206b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:03.330488 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-507740b4-0544-4ff7-847a-158969db206b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:03.345355 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507740b4-0544-4ff7-847a-158969db206b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:03.355386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b65a27-f141-47d0-a1cc-dd1dc1c0adc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:03.356512 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b65a27-f141-47d0-a1cc-dd1dc1c0adc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:03.356947 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b65a27-f141-47d0-a1cc-dd1dc1c0adc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:03.357574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b65a27-f141-47d0-a1cc-dd1dc1c0adc2 admin admin] PATH_INFO: `/v3/users/22efe134e4534ee78456833b4b690f88` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:03.359412 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b65a27-f141-47d0-a1cc-dd1dc1c0adc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22efe134e4534ee78456833b4b690f88)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:03.361860 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b65a27-f141-47d0-a1cc-dd1dc1c0adc2 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:03.376404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f3bef-7091-4b95-86a1-3ab2f3ae440f admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:03.380381 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf9f462-4eeb-476d-9b01-72ea9a51ba7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:03.381053 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9f462-4eeb-476d-9b01-72ea9a51ba7f service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:03.381345 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9f462-4eeb-476d-9b01-72ea9a51ba7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:03.381630 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf9f462-4eeb-476d-9b01-72ea9a51ba7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:03.397403 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf9f462-4eeb-476d-9b01-72ea9a51ba7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:03.400292 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf9f462-4eeb-476d-9b01-72ea9a51ba7f service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:03.412241 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1017/2034] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:35:03.642401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-51b65a27-f141-47d0-a1cc-dd1dc1c0adc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22efe134e4534ee78456833b4b690f88'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:03.643096 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-51b65a27-f141-47d0-a1cc-dd1dc1c0adc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22efe134e4534ee78456833b4b690f88'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:03.661560 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-51b65a27-f141-47d0-a1cc-dd1dc1c0adc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22efe134e4534ee78456833b4b690f88'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:03.676076 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-51b65a27-f141-47d0-a1cc-dd1dc1c0adc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22efe134e4534ee78456833b4b690f88'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:03.773642 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-51b65a27-f141-47d0-a1cc-dd1dc1c0adc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22efe134e4534ee78456833b4b690f88'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:03.822178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1018/2035] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:35:03 2022] DELETE /identity/v3/users/22efe134e4534ee78456833b4b690f88 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:35:03.826841 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7ab7f-77c1-4f20-a19a-997915540e0b None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:03.827393 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7ab7f-77c1-4f20-a19a-997915540e0b None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:03.828071 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e7ab7f-77c1-4f20-a19a-997915540e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:03.866243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-75e7ab7f-77c1-4f20-a19a-997915540e0b None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:03.869865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-75e7ab7f-77c1-4f20-a19a-997915540e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:03.892332 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:03.948616 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1018/2036] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:03.969427 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b65a27-f141-47d0-a1cc-dd1dc1c0adc2 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:03.971106 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-51b65a27-f141-47d0-a1cc-dd1dc1c0adc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:03.974183 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e7ab7f-77c1-4f20-a19a-997915540e0b None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:03.979212 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-75e7ab7f-77c1-4f20-a19a-997915540e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:03.987432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b65a27-f141-47d0-a1cc-dd1dc1c0adc2 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:03.995227 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d11f0-4dbf-4b3f-87ec-a454cc5cb3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:03.995750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d11f0-4dbf-4b3f-87ec-a454cc5cb3f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:03.996097 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d11f0-4dbf-4b3f-87ec-a454cc5cb3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:03.996570 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9d11f0-4dbf-4b3f-87ec-a454cc5cb3f9 admin admin] PATH_INFO: `/v3/projects/0c97f86db9ec4d01b5a0e2de5978a51c` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:03.998057 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9d11f0-4dbf-4b3f-87ec-a454cc5cb3f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c97f86db9ec4d01b5a0e2de5978a51c)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:04.000187 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9d11f0-4dbf-4b3f-87ec-a454cc5cb3f9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:04.068216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e7ab7f-77c1-4f20-a19a-997915540e0b None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:04.072725 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62a620-5e3e-436e-b081-32d4824be682 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:04.073440 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62a620-5e3e-436e-b081-32d4824be682 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:04.073731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62a620-5e3e-436e-b081-32d4824be682 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:04.074012 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62a620-5e3e-436e-b081-32d4824be682 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:04.093693 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac62a620-5e3e-436e-b081-32d4824be682 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:04.096151 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac62a620-5e3e-436e-b081-32d4824be682 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:04.114026 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1019/2037] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:35:04.177161 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:04.186254 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0e9d11f0-4dbf-4b3f-87ec-a454cc5cb3f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c97f86db9ec4d01b5a0e2de5978a51c', 'request_id': 'req-0e9d11f0-4dbf-4b3f-87ec-a454cc5cb3f9'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:04.293825 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0e9d11f0-4dbf-4b3f-87ec-a454cc5cb3f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c97f86db9ec4d01b5a0e2de5978a51c', 'request_id': 'req-0e9d11f0-4dbf-4b3f-87ec-a454cc5cb3f9'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:04.300260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0e9d11f0-4dbf-4b3f-87ec-a454cc5cb3f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c97f86db9ec4d01b5a0e2de5978a51c', 'request_id': 'req-0e9d11f0-4dbf-4b3f-87ec-a454cc5cb3f9'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:04.314422 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0e9d11f0-4dbf-4b3f-87ec-a454cc5cb3f9 admin admin] The token cache is being invalidate because project 0c97f86db9ec4d01b5a0e2de5978a51c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:35:04.314950 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0e9d11f0-4dbf-4b3f-87ec-a454cc5cb3f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:04.315820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0e9d11f0-4dbf-4b3f-87ec-a454cc5cb3f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:04.321676 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1019/2038] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:35:03 2022] DELETE /identity/v3/projects/0c97f86db9ec4d01b5a0e2de5978a51c => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:04.322601 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62a620-5e3e-436e-b081-32d4824be682 service neutron] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:04.334858 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62a620-5e3e-436e-b081-32d4824be682 service neutron] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:04.430944 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0fa16-1e35-46c6-b6ad-c5cf3d18f43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:04.433105 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0fa16-1e35-46c6-b6ad-c5cf3d18f43e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:04.433611 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0fa16-1e35-46c6-b6ad-c5cf3d18f43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:04.438459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b0fa16-1e35-46c6-b6ad-c5cf3d18f43e admin admin] PATH_INFO: `/v3/users/0133caf8926746e48ed053076551421e` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:04.438459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b0fa16-1e35-46c6-b6ad-c5cf3d18f43e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0133caf8926746e48ed053076551421e)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:04.438459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b0fa16-1e35-46c6-b6ad-c5cf3d18f43e admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:04.695485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-63b0fa16-1e35-46c6-b6ad-c5cf3d18f43e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0133caf8926746e48ed053076551421e'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:04.710162 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d11f0-4dbf-4b3f-87ec-a454cc5cb3f9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:04.712252 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0e9d11f0-4dbf-4b3f-87ec-a454cc5cb3f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:04.749053 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9d11f0-4dbf-4b3f-87ec-a454cc5cb3f9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:04.751673 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7bffc-beee-4b7d-8484-e9cfcb13295c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:04.752714 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7bffc-beee-4b7d-8484-e9cfcb13295c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:04.753110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7bffc-beee-4b7d-8484-e9cfcb13295c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:04.753187 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d7bffc-beee-4b7d-8484-e9cfcb13295c admin admin] PATH_INFO: `/v3/projects/d717eb7df90542af8edd4610e7e4d4d0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:04.754543 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7bffc-beee-4b7d-8484-e9cfcb13295c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d717eb7df90542af8edd4610e7e4d4d0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:04.757423 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d7bffc-beee-4b7d-8484-e9cfcb13295c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:04.785259 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-63b0fa16-1e35-46c6-b6ad-c5cf3d18f43e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0133caf8926746e48ed053076551421e'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:04.801913 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:04.808981 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-63b0fa16-1e35-46c6-b6ad-c5cf3d18f43e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0133caf8926746e48ed053076551421e'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:04.810152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-63b0fa16-1e35-46c6-b6ad-c5cf3d18f43e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0133caf8926746e48ed053076551421e'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:04.810937 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-63b0fa16-1e35-46c6-b6ad-c5cf3d18f43e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0133caf8926746e48ed053076551421e'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:04.834973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1020/2039] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:35:04 2022] DELETE /identity/v3/users/0133caf8926746e48ed053076551421e => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:35:04.939334 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-72d7bffc-beee-4b7d-8484-e9cfcb13295c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd717eb7df90542af8edd4610e7e4d4d0', 'request_id': 'req-72d7bffc-beee-4b7d-8484-e9cfcb13295c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:05.082460 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-72d7bffc-beee-4b7d-8484-e9cfcb13295c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd717eb7df90542af8edd4610e7e4d4d0', 'request_id': 'req-72d7bffc-beee-4b7d-8484-e9cfcb13295c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:05.093812 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-72d7bffc-beee-4b7d-8484-e9cfcb13295c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd717eb7df90542af8edd4610e7e4d4d0', 'request_id': 'req-72d7bffc-beee-4b7d-8484-e9cfcb13295c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:05.108102 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-72d7bffc-beee-4b7d-8484-e9cfcb13295c admin admin] The token cache is being invalidate because project d717eb7df90542af8edd4610e7e4d4d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:35:05.108321 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-72d7bffc-beee-4b7d-8484-e9cfcb13295c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:05.109254 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-72d7bffc-beee-4b7d-8484-e9cfcb13295c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:05.113840 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1020/2040] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:35:04 2022] DELETE /identity/v3/projects/d717eb7df90542af8edd4610e7e4d4d0 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:05.116453 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0fa16-1e35-46c6-b6ad-c5cf3d18f43e admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:05.123331 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-63b0fa16-1e35-46c6-b6ad-c5cf3d18f43e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:05.223201 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b0fa16-1e35-46c6-b6ad-c5cf3d18f43e admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:05.324202 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253267a8-2a0e-4c1b-9ad9-0aa6f40e927e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:05.325201 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253267a8-2a0e-4c1b-9ad9-0aa6f40e927e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:05.325786 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253267a8-2a0e-4c1b-9ad9-0aa6f40e927e admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:05.332248 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253267a8-2a0e-4c1b-9ad9-0aa6f40e927e admin admin] PATH_INFO: `/v3/users/10e7373619ed483db3030760aa0e695e` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:05.332248 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253267a8-2a0e-4c1b-9ad9-0aa6f40e927e admin admin] RBAC: Authorizing `identity:delete_user(user_id=10e7373619ed483db3030760aa0e695e)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:05.332248 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253267a8-2a0e-4c1b-9ad9-0aa6f40e927e admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:05.408601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7bffc-beee-4b7d-8484-e9cfcb13295c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:05.408601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-72d7bffc-beee-4b7d-8484-e9cfcb13295c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:05.422096 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d7bffc-beee-4b7d-8484-e9cfcb13295c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:05.426820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c45127-9a5e-4773-a21f-d594a401bce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:05.427738 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c45127-9a5e-4773-a21f-d594a401bce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:05.428031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c45127-9a5e-4773-a21f-d594a401bce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:05.428314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c45127-9a5e-4773-a21f-d594a401bce7 admin admin] PATH_INFO: `/v3/projects/c85724bcfa2644b78244b417d8dac24f` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:05.433615 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c45127-9a5e-4773-a21f-d594a401bce7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c85724bcfa2644b78244b417d8dac24f)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:05.435637 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c45127-9a5e-4773-a21f-d594a401bce7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:05.547358 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-253267a8-2a0e-4c1b-9ad9-0aa6f40e927e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10e7373619ed483db3030760aa0e695e'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:05.624336 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-253267a8-2a0e-4c1b-9ad9-0aa6f40e927e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10e7373619ed483db3030760aa0e695e'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:05.648318 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b9c45127-9a5e-4773-a21f-d594a401bce7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c85724bcfa2644b78244b417d8dac24f', 'request_id': 'req-b9c45127-9a5e-4773-a21f-d594a401bce7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:05.661807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-253267a8-2a0e-4c1b-9ad9-0aa6f40e927e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10e7373619ed483db3030760aa0e695e'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:05.663136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-253267a8-2a0e-4c1b-9ad9-0aa6f40e927e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10e7373619ed483db3030760aa0e695e'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:05.666034 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-253267a8-2a0e-4c1b-9ad9-0aa6f40e927e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10e7373619ed483db3030760aa0e695e'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:05.690331 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1021/2041] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:35:05 2022] DELETE /identity/v3/users/10e7373619ed483db3030760aa0e695e => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:05.779524 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b9c45127-9a5e-4773-a21f-d594a401bce7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c85724bcfa2644b78244b417d8dac24f', 'request_id': 'req-b9c45127-9a5e-4773-a21f-d594a401bce7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:05.789737 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b9c45127-9a5e-4773-a21f-d594a401bce7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c85724bcfa2644b78244b417d8dac24f', 'request_id': 'req-b9c45127-9a5e-4773-a21f-d594a401bce7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:05.815660 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b9c45127-9a5e-4773-a21f-d594a401bce7 admin admin] The token cache is being invalidate because project c85724bcfa2644b78244b417d8dac24f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:35:05.816160 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b9c45127-9a5e-4773-a21f-d594a401bce7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:05.816693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b9c45127-9a5e-4773-a21f-d594a401bce7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:05.818360 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1021/2042] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:35:05 2022] DELETE /identity/v3/projects/c85724bcfa2644b78244b417d8dac24f => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:05.823111 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253267a8-2a0e-4c1b-9ad9-0aa6f40e927e admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:05.826419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-253267a8-2a0e-4c1b-9ad9-0aa6f40e927e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:05.837514 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:05.892641 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253267a8-2a0e-4c1b-9ad9-0aa6f40e927e admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:05.897183 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977910c3-18f0-4262-b751-dd65cd113768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:05.898064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977910c3-18f0-4262-b751-dd65cd113768 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:05.898344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977910c3-18f0-4262-b751-dd65cd113768 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:05.898615 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977910c3-18f0-4262-b751-dd65cd113768 admin admin] PATH_INFO: `/v3/users/4c06d450248b4884a51bddddc9e1c6f3` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:05.899932 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977910c3-18f0-4262-b751-dd65cd113768 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c06d450248b4884a51bddddc9e1c6f3)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:05.901885 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977910c3-18f0-4262-b751-dd65cd113768 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:06.175593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-977910c3-18f0-4262-b751-dd65cd113768 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c06d450248b4884a51bddddc9e1c6f3'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:06.260917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-977910c3-18f0-4262-b751-dd65cd113768 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c06d450248b4884a51bddddc9e1c6f3'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:06.278161 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-977910c3-18f0-4262-b751-dd65cd113768 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c06d450248b4884a51bddddc9e1c6f3'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:06.278945 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-977910c3-18f0-4262-b751-dd65cd113768 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c06d450248b4884a51bddddc9e1c6f3'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:06.280325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-977910c3-18f0-4262-b751-dd65cd113768 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c06d450248b4884a51bddddc9e1c6f3'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:06.313580 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1022/2043] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:35:05 2022] DELETE /identity/v3/users/4c06d450248b4884a51bddddc9e1c6f3 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:06.400834 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c45127-9a5e-4773-a21f-d594a401bce7 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:06.402305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b9c45127-9a5e-4773-a21f-d594a401bce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:06.474710 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=18,keystone:SELECT=79,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:06.498328 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c45127-9a5e-4773-a21f-d594a401bce7 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:06.562913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b50a04-700c-4da4-b144-fe81c8c12f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:06.563794 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b50a04-700c-4da4-b144-fe81c8c12f17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:06.564082 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b50a04-700c-4da4-b144-fe81c8c12f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:06.564370 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b50a04-700c-4da4-b144-fe81c8c12f17 admin admin] PATH_INFO: `/v3/projects/07d3e67c87d945b7aa3d135625239925` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:06.565658 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b50a04-700c-4da4-b144-fe81c8c12f17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07d3e67c87d945b7aa3d135625239925)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:06.567816 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b50a04-700c-4da4-b144-fe81c8c12f17 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:06.788480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c1b50a04-700c-4da4-b144-fe81c8c12f17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07d3e67c87d945b7aa3d135625239925', 'request_id': 'req-c1b50a04-700c-4da4-b144-fe81c8c12f17'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:06.868603 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977910c3-18f0-4262-b751-dd65cd113768 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:06.870117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-977910c3-18f0-4262-b751-dd65cd113768 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:06.884891 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977910c3-18f0-4262-b751-dd65cd113768 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:06.893424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c1b50a04-700c-4da4-b144-fe81c8c12f17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07d3e67c87d945b7aa3d135625239925', 'request_id': 'req-c1b50a04-700c-4da4-b144-fe81c8c12f17'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:06.903316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c1b50a04-700c-4da4-b144-fe81c8c12f17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07d3e67c87d945b7aa3d135625239925', 'request_id': 'req-c1b50a04-700c-4da4-b144-fe81c8c12f17'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:06.926975 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c1b50a04-700c-4da4-b144-fe81c8c12f17 admin admin] The token cache is being invalidate because project 07d3e67c87d945b7aa3d135625239925 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:35:06.927369 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c1b50a04-700c-4da4-b144-fe81c8c12f17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:06.930314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c1b50a04-700c-4da4-b144-fe81c8c12f17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:06.930314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1022/2044] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:35:06 2022] DELETE /identity/v3/projects/07d3e67c87d945b7aa3d135625239925 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:06.935423 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddd311-f21b-437f-bd35-2c0552226b90 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:06.936010 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddd311-f21b-437f-bd35-2c0552226b90 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:06.936918 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cddd311-f21b-437f-bd35-2c0552226b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:06.964150 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6cddd311-f21b-437f-bd35-2c0552226b90 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:06.965614 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6cddd311-f21b-437f-bd35-2c0552226b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:06.975522 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b80561-c47d-476e-80a5-19c847fc8c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:06.976731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b80561-c47d-476e-80a5-19c847fc8c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:06.978960 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b80561-c47d-476e-80a5-19c847fc8c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:06.979453 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b80561-c47d-476e-80a5-19c847fc8c39 admin admin] PATH_INFO: `/v3/projects/21d082d5c47446abb54ca92d1ab88421` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:06.981685 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b80561-c47d-476e-80a5-19c847fc8c39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21d082d5c47446abb54ca92d1ab88421)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:06.983800 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b80561-c47d-476e-80a5-19c847fc8c39 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:07.035690 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1023/2045] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:07.043584 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69dab2-55e9-43b3-bde6-3ed654d2b202 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:07.043917 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69dab2-55e9-43b3-bde6-3ed654d2b202 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:07.044136 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69dab2-55e9-43b3-bde6-3ed654d2b202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:07.069533 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7c69dab2-55e9-43b3-bde6-3ed654d2b202 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:07.069533 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7c69dab2-55e9-43b3-bde6-3ed654d2b202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:07.080155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1024/2046] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:35:07.084729 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce4ef1-31e9-4613-af68-51d9c3e51317 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:07.085487 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce4ef1-31e9-4613-af68-51d9c3e51317 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:07.085597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce4ef1-31e9-4613-af68-51d9c3e51317 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:07.090371 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1025/2047] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jun 30 10:35:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:35:07.090371 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce4ef1-31e9-4613-af68-51d9c3e51317 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:07.108480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce4ef1-31e9-4613-af68-51d9c3e51317 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:07.117010 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3ca27-20c6-48c7-8d2a-97ac0c26f590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:07.131684 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3ca27-20c6-48c7-8d2a-97ac0c26f590 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:07.131684 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3ca27-20c6-48c7-8d2a-97ac0c26f590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:07.131684 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3ca27-20c6-48c7-8d2a-97ac0c26f590 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:07.131684 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da3ca27-20c6-48c7-8d2a-97ac0c26f590 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:35:07.131684 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da3ca27-20c6-48c7-8d2a-97ac0c26f590 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:07.131684 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da3ca27-20c6-48c7-8d2a-97ac0c26f590 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:07.137910 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1026/2048] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:35:07.139458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf6b36-7811-4519-b3a9-53033c9a47b7 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:07.139750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf6b36-7811-4519-b3a9-53033c9a47b7 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:07.140032 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bf6b36-7811-4519-b3a9-53033c9a47b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:07.143429 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:07.157439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-65b80561-c47d-476e-80a5-19c847fc8c39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21d082d5c47446abb54ca92d1ab88421', 'request_id': 'req-65b80561-c47d-476e-80a5-19c847fc8c39'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:07.165353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-15bf6b36-7811-4519-b3a9-53033c9a47b7 None None] MFA Rules not processed for user `46c99f43a9344d7d92632990e45bb9cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:07.166587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-15bf6b36-7811-4519-b3a9-53033c9a47b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:07.173274 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-65b80561-c47d-476e-80a5-19c847fc8c39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21d082d5c47446abb54ca92d1ab88421', 'request_id': 'req-65b80561-c47d-476e-80a5-19c847fc8c39'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:07.179515 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-65b80561-c47d-476e-80a5-19c847fc8c39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21d082d5c47446abb54ca92d1ab88421', 'request_id': 'req-65b80561-c47d-476e-80a5-19c847fc8c39'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:07.247427 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1027/2049] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jun 30 10:35:07 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:07.252182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf6b36-7811-4519-b3a9-53033c9a47b7 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:07.252182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:07.319068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bf6b36-7811-4519-b3a9-53033c9a47b7 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:07.324478 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649567f4-24d3-4c9e-ac7a-c9658997a0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:07.325142 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649567f4-24d3-4c9e-ac7a-c9658997a0bc admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:07.325429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649567f4-24d3-4c9e-ac7a-c9658997a0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:07.325708 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649567f4-24d3-4c9e-ac7a-c9658997a0bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:07.326406 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649567f4-24d3-4c9e-ac7a-c9658997a0bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:07.328785 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649567f4-24d3-4c9e-ac7a-c9658997a0bc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:07.340052 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-65b80561-c47d-476e-80a5-19c847fc8c39 admin admin] The token cache is being invalidate because project 21d082d5c47446abb54ca92d1ab88421 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:35:07.340694 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-65b80561-c47d-476e-80a5-19c847fc8c39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:07.341371 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-65b80561-c47d-476e-80a5-19c847fc8c39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:07.343314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1023/2050] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:35:06 2022] DELETE /identity/v3/projects/21d082d5c47446abb54ca92d1ab88421 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:35:07.348287 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b80561-c47d-476e-80a5-19c847fc8c39 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:07.349762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-65b80561-c47d-476e-80a5-19c847fc8c39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:07.362457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b80561-c47d-476e-80a5-19c847fc8c39 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:07.372578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94720fbb-7f21-47d4-84e1-861d71af837d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:07.373488 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94720fbb-7f21-47d4-84e1-861d71af837d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:07.373772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94720fbb-7f21-47d4-84e1-861d71af837d admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:07.374049 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94720fbb-7f21-47d4-84e1-861d71af837d admin admin] PATH_INFO: `/v3/users/9aa86b4d8a924a068f6b4da6dbf5c6bd` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:07.375309 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94720fbb-7f21-47d4-84e1-861d71af837d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aa86b4d8a924a068f6b4da6dbf5c6bd)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:07.377486 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94720fbb-7f21-47d4-84e1-861d71af837d admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:07.439072 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1028/2051] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:35:07 2022] POST /identity/v3/projects => generated 361 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:07.439963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649567f4-24d3-4c9e-ac7a-c9658997a0bc admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:07.441536 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-649567f4-24d3-4c9e-ac7a-c9658997a0bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:07.517747 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649567f4-24d3-4c9e-ac7a-c9658997a0bc admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:07.524089 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2d576-31bb-474e-b504-b10a998a82c7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46c99f43a9344d7d92632990e45bb9cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:07.524803 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2d576-31bb-474e-b504-b10a998a82c7 service placement] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:07.525120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2d576-31bb-474e-b504-b10a998a82c7 service placement] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:07.525587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2d576-31bb-474e-b504-b10a998a82c7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:07.527875 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f0d2d576-31bb-474e-b504-b10a998a82c7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:07.542819 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d2d576-31bb-474e-b504-b10a998a82c7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:07.544507 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d2d576-31bb-474e-b504-b10a998a82c7 service placement] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:07.560386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1029/2052] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:35:07.560386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c057b07-367f-4d0c-8766-3ab35d1fdb30 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:07.560386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c057b07-367f-4d0c-8766-3ab35d1fdb30 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:07.560386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c057b07-367f-4d0c-8766-3ab35d1fdb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:07.579518 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5c057b07-367f-4d0c-8766-3ab35d1fdb30 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:07.581153 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5c057b07-367f-4d0c-8766-3ab35d1fdb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:07.622296 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-94720fbb-7f21-47d4-84e1-861d71af837d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aa86b4d8a924a068f6b4da6dbf5c6bd'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:07.645829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1030/2053] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:07.653245 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c057b07-367f-4d0c-8766-3ab35d1fdb30 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:07.654905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5c057b07-367f-4d0c-8766-3ab35d1fdb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:07.671142 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c057b07-367f-4d0c-8766-3ab35d1fdb30 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:07.675924 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2e0e6-afa3-4020-bc02-4491ac7a4de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:07.676593 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2e0e6-afa3-4020-bc02-4491ac7a4de9 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:07.676889 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2e0e6-afa3-4020-bc02-4491ac7a4de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:07.677153 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d2e0e6-afa3-4020-bc02-4491ac7a4de9 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:07.677840 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d2e0e6-afa3-4020-bc02-4491ac7a4de9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:07.679796 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d2e0e6-afa3-4020-bc02-4491ac7a4de9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:07.704771 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-94720fbb-7f21-47d4-84e1-861d71af837d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aa86b4d8a924a068f6b4da6dbf5c6bd'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:07.724268 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-94720fbb-7f21-47d4-84e1-861d71af837d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aa86b4d8a924a068f6b4da6dbf5c6bd'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:07.725019 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-94720fbb-7f21-47d4-84e1-861d71af837d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aa86b4d8a924a068f6b4da6dbf5c6bd'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:07.725647 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-94720fbb-7f21-47d4-84e1-861d71af837d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aa86b4d8a924a068f6b4da6dbf5c6bd'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:07.739027 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1024/2054] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:35:07 2022] DELETE /identity/v3/users/9aa86b4d8a924a068f6b4da6dbf5c6bd => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:07.740503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0691e51-1033-4bc4-9642-1b225c442091 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:07.740794 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0691e51-1033-4bc4-9642-1b225c442091 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:07.741077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0691e51-1033-4bc4-9642-1b225c442091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:07.763596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-d0691e51-1033-4bc4-9642-1b225c442091 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:07.764838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d0691e51-1033-4bc4-9642-1b225c442091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:07.785357 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1031/2055] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:35:07 2022] POST /identity/v3/users => generated 281 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:07.793344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2e0e6-afa3-4020-bc02-4491ac7a4de9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:07.795366 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-11d2e0e6-afa3-4020-bc02-4491ac7a4de9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:07.809141 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d2e0e6-afa3-4020-bc02-4491ac7a4de9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:07.834633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1025/2056] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:07.843668 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0691e51-1033-4bc4-9642-1b225c442091 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:07.853407 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0691e51-1033-4bc4-9642-1b225c442091 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:07.859766 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c050ef-8026-4a00-8092-321a941a73fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:07.860447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c050ef-8026-4a00-8092-321a941a73fe admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:07.860734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c050ef-8026-4a00-8092-321a941a73fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:07.861061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c050ef-8026-4a00-8092-321a941a73fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:07.861799 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c050ef-8026-4a00-8092-321a941a73fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:35:07.862109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c050ef-8026-4a00-8092-321a941a73fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:07.863763 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c050ef-8026-4a00-8092-321a941a73fe admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:07.887492 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1026/2057] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:35:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:35:07.891495 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c050ef-8026-4a00-8092-321a941a73fe admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:07.896968 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c2cb7-6140-4484-a695-731fba045de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:07.897768 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c2cb7-6140-4484-a695-731fba045de9 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:07.898151 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c2cb7-6140-4484-a695-731fba045de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:07.898348 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c2cb7-6140-4484-a695-731fba045de9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:07.899421 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c2cb7-6140-4484-a695-731fba045de9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:07.900985 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c2cb7-6140-4484-a695-731fba045de9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:07.915111 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c050ef-8026-4a00-8092-321a941a73fe admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:07.917098 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1032/2058] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:35:07.918047 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c2cb7-6140-4484-a695-731fba045de9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:07.922554 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6516f4-77c9-4837-aa81-a55377a31307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:07.923231 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6516f4-77c9-4837-aa81-a55377a31307 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:07.923551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6516f4-77c9-4837-aa81-a55377a31307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:07.927199 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6516f4-77c9-4837-aa81-a55377a31307 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:07.929363 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6516f4-77c9-4837-aa81-a55377a31307 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:07.933222 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c2cb7-6140-4484-a695-731fba045de9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:07.943305 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6516f4-77c9-4837-aa81-a55377a31307 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:07.943468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c28c3-daed-4c20-bc74-f546d77177f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:07.943468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c28c3-daed-4c20-bc74-f546d77177f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:07.943468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c28c3-daed-4c20-bc74-f546d77177f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:07.943468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c28c3-daed-4c20-bc74-f546d77177f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:07.943824 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c28c3-daed-4c20-bc74-f546d77177f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:07.943824 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c28c3-daed-4c20-bc74-f546d77177f8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:07.974324 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1033/2059] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:07 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:35:07.978545 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c28c3-daed-4c20-bc74-f546d77177f8 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:07.982355 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=82,keystone:DELETE=15,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:07.995310 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c28c3-daed-4c20-bc74-f546d77177f8 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:08.000839 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d597d-e6d2-491d-834c-da654f580099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:08.001754 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d597d-e6d2-491d-834c-da654f580099 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:08.002045 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d597d-e6d2-491d-834c-da654f580099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:08.002324 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d597d-e6d2-491d-834c-da654f580099 admin admin] PATH_INFO: `/v3/projects/a5f9ffff0c414642a9fed631fbf111d4/users/23c6c70a0cb84dc4903c042afb441e2a/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:08.020035 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d597d-e6d2-491d-834c-da654f580099 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5f9ffff0c414642a9fed631fbf111d4, user_id=23c6c70a0cb84dc4903c042afb441e2a, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:08.022731 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d597d-e6d2-491d-834c-da654f580099 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:08.042374 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:08.109152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1027/2060] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:35:07 2022] POST /identity/v3/projects => generated 383 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:08.116245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6516f4-77c9-4837-aa81-a55377a31307 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:08.144907 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1034/2061] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:35:07 2022] PUT /identity/v3/projects/a5f9ffff0c414642a9fed631fbf111d4/users/23c6c70a0cb84dc4903c042afb441e2a/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:35:08.160086 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d597d-e6d2-491d-834c-da654f580099 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:08.175289 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f62d597d-e6d2-491d-834c-da654f580099 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:08.206880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6516f4-77c9-4837-aa81-a55377a31307 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:08.252500 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d597d-e6d2-491d-834c-da654f580099 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:08.257815 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06daf8-13be-4a4c-890a-a4113fdbf547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:08.258906 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06daf8-13be-4a4c-890a-a4113fdbf547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:08.259203 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06daf8-13be-4a4c-890a-a4113fdbf547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:08.259513 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06daf8-13be-4a4c-890a-a4113fdbf547 admin admin] PATH_INFO: `/v3/projects/c96cea5375d54cb0aa93cdc2f24f150a` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:08.260904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06daf8-13be-4a4c-890a-a4113fdbf547 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c96cea5375d54cb0aa93cdc2f24f150a)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:08.262879 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06daf8-13be-4a4c-890a-a4113fdbf547 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:08.277949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368fc8c2-fd3b-4941-b042-6b95084fb082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:08.278643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fc8c2-fd3b-4941-b042-6b95084fb082 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:08.278975 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fc8c2-fd3b-4941-b042-6b95084fb082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:08.279256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fc8c2-fd3b-4941-b042-6b95084fb082 admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:08.280284 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368fc8c2-fd3b-4941-b042-6b95084fb082 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:08.282456 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368fc8c2-fd3b-4941-b042-6b95084fb082 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:08.382837 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1028/2062] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:35:08 2022] POST /identity/v3/users => generated 292 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:35:08.382837 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2805b3-1028-457c-abb7-f3d55fd34ef5 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:08.382837 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2805b3-1028-457c-abb7-f3d55fd34ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:08.382837 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2805b3-1028-457c-abb7-f3d55fd34ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:08.391582 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7a06daf8-13be-4a4c-890a-a4113fdbf547 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c96cea5375d54cb0aa93cdc2f24f150a', 'request_id': 'req-7a06daf8-13be-4a4c-890a-a4113fdbf547'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:08.410930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-bb2805b3-1028-457c-abb7-f3d55fd34ef5 None None] MFA Rules not processed for user `23c6c70a0cb84dc4903c042afb441e2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:08.412215 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-bb2805b3-1028-457c-abb7-f3d55fd34ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:08.475471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7a06daf8-13be-4a4c-890a-a4113fdbf547 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c96cea5375d54cb0aa93cdc2f24f150a', 'request_id': 'req-7a06daf8-13be-4a4c-890a-a4113fdbf547'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:08.476929 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1029/2063] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:08 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:08.477787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2805b3-1028-457c-abb7-f3d55fd34ef5 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:08.485746 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7a06daf8-13be-4a4c-890a-a4113fdbf547 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c96cea5375d54cb0aa93cdc2f24f150a', 'request_id': 'req-7a06daf8-13be-4a4c-890a-a4113fdbf547'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:08.497455 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7a06daf8-13be-4a4c-890a-a4113fdbf547 admin admin] The token cache is being invalidate because project c96cea5375d54cb0aa93cdc2f24f150a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:35:08.497750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7a06daf8-13be-4a4c-890a-a4113fdbf547 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:08.498468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7a06daf8-13be-4a4c-890a-a4113fdbf547 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:08.500094 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1035/2064] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:35:08 2022] DELETE /identity/v3/projects/c96cea5375d54cb0aa93cdc2f24f150a => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:08.501069 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06daf8-13be-4a4c-890a-a4113fdbf547 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:08.502555 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7a06daf8-13be-4a4c-890a-a4113fdbf547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:08.540764 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:08.546402 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2805b3-1028-457c-abb7-f3d55fd34ef5 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:08.551091 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9789d0b-1940-4a39-bfc2-85acaad5265a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:08.551787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9789d0b-1940-4a39-bfc2-85acaad5265a admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:08.552071 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9789d0b-1940-4a39-bfc2-85acaad5265a admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:08.552343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9789d0b-1940-4a39-bfc2-85acaad5265a admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:08.553070 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9789d0b-1940-4a39-bfc2-85acaad5265a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:08.554728 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9789d0b-1940-4a39-bfc2-85acaad5265a admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:08.565564 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1030/2065] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:35:08.566968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43436924-4e56-4999-8444-57459838f8f4 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:08.567259 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43436924-4e56-4999-8444-57459838f8f4 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:08.567608 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43436924-4e56-4999-8444-57459838f8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:08.576079 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06daf8-13be-4a4c-890a-a4113fdbf547 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:08.580980 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43278030-4c26-47a9-8874-d28a862b4087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:08.581649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43278030-4c26-47a9-8874-d28a862b4087 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:08.581937 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43278030-4c26-47a9-8874-d28a862b4087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:08.582219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43278030-4c26-47a9-8874-d28a862b4087 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:08.585844 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43278030-4c26-47a9-8874-d28a862b4087 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:08.585844 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43278030-4c26-47a9-8874-d28a862b4087 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:08.588913 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-43436924-4e56-4999-8444-57459838f8f4 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:08.590148 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-43436924-4e56-4999-8444-57459838f8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:08.598102 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1031/2066] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:08.598878 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43436924-4e56-4999-8444-57459838f8f4 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:08.600239 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-43436924-4e56-4999-8444-57459838f8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:08.613379 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43436924-4e56-4999-8444-57459838f8f4 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:08.618601 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fc0766-3ea9-43f2-a934-f52871568fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:08.619330 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc0766-3ea9-43f2-a934-f52871568fe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:08.619613 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc0766-3ea9-43f2-a934-f52871568fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:08.622185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fc0766-3ea9-43f2-a934-f52871568fe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:08.622185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc0766-3ea9-43f2-a934-f52871568fe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:08.622185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fc0766-3ea9-43f2-a934-f52871568fe3 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:08.631951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1032/2067] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:35:08.633123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceff110-b922-4c4d-9684-77954ba5f2c9 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:08.634471 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceff110-b922-4c4d-9684-77954ba5f2c9 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:08.634870 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceff110-b922-4c4d-9684-77954ba5f2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:08.653157 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-8ceff110-b922-4c4d-9684-77954ba5f2c9 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:08.654336 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-8ceff110-b922-4c4d-9684-77954ba5f2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:08.662514 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1033/2068] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:08.663247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceff110-b922-4c4d-9684-77954ba5f2c9 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:08.673784 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceff110-b922-4c4d-9684-77954ba5f2c9 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:08.678935 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a268dcc-c253-48ac-9e5b-f1b8e1375d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:08.679713 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1036/2069] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:35:08 2022] POST /identity/v3/projects => generated 363 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:08.692799 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a268dcc-c253-48ac-9e5b-f1b8e1375d26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:08.692799 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a268dcc-c253-48ac-9e5b-f1b8e1375d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:08.692799 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a268dcc-c253-48ac-9e5b-f1b8e1375d26 admin admin] PATH_INFO: `/v3/projects/9f0377219b474143b09431f000d38f28/users/e9aac5f2e6a7416fb49608f0bd21c0a2/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:08.693126 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43278030-4c26-47a9-8874-d28a862b4087 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:08.699358 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a268dcc-c253-48ac-9e5b-f1b8e1375d26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f0377219b474143b09431f000d38f28, user_id=e9aac5f2e6a7416fb49608f0bd21c0a2, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:08.702061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a268dcc-c253-48ac-9e5b-f1b8e1375d26 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:08.709776 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:08.744790 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43278030-4c26-47a9-8874-d28a862b4087 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:08.749531 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7909949-b88c-4a1b-b0c7-04d9b55ac355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:08.750371 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7909949-b88c-4a1b-b0c7-04d9b55ac355 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:08.750657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7909949-b88c-4a1b-b0c7-04d9b55ac355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:08.750934 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7909949-b88c-4a1b-b0c7-04d9b55ac355 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:08.751704 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7909949-b88c-4a1b-b0c7-04d9b55ac355 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:35:08.752017 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7909949-b88c-4a1b-b0c7-04d9b55ac355 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:08.753629 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7909949-b88c-4a1b-b0c7-04d9b55ac355 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:08.765962 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1037/2070] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:35:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:35:08.766729 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7909949-b88c-4a1b-b0c7-04d9b55ac355 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:08.778473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7909949-b88c-4a1b-b0c7-04d9b55ac355 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:08.783600 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ecfefe-f737-472b-b00c-d8fa712ae6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:08.784263 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecfefe-f737-472b-b00c-d8fa712ae6b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:08.784630 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecfefe-f737-472b-b00c-d8fa712ae6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:08.784923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ecfefe-f737-472b-b00c-d8fa712ae6b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:08.785595 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ecfefe-f737-472b-b00c-d8fa712ae6b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:08.787669 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ecfefe-f737-472b-b00c-d8fa712ae6b1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:08.808268 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1034/2071] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:35:08 2022] PUT /identity/v3/projects/9f0377219b474143b09431f000d38f28/users/e9aac5f2e6a7416fb49608f0bd21c0a2/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:08.809114 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a268dcc-c253-48ac-9e5b-f1b8e1375d26 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:08.872516 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a268dcc-c253-48ac-9e5b-f1b8e1375d26 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:08.877512 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0ce79-2cd1-42a0-9466-4e2c0ca83cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:08.878181 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0ce79-2cd1-42a0-9466-4e2c0ca83cc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:08.878465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0ce79-2cd1-42a0-9466-4e2c0ca83cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:08.878739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0ce79-2cd1-42a0-9466-4e2c0ca83cc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:08.879464 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e0ce79-2cd1-42a0-9466-4e2c0ca83cc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:08.881445 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e0ce79-2cd1-42a0-9466-4e2c0ca83cc9 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:08.890065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1038/2072] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:35:08 2022] POST /identity/v3/users => generated 288 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:08.893700 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44538e57-0101-4c9f-bdfe-6db809bea61a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:08.893700 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44538e57-0101-4c9f-bdfe-6db809bea61a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:08.893700 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44538e57-0101-4c9f-bdfe-6db809bea61a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:08.920478 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-44538e57-0101-4c9f-bdfe-6db809bea61a None None] MFA Rules not processed for user `e9aac5f2e6a7416fb49608f0bd21c0a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:08.921682 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-44538e57-0101-4c9f-bdfe-6db809bea61a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:08.987897 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1035/2073] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:35:08 2022] POST /identity/v3/projects => generated 373 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:08.992472 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0ce79-2cd1-42a0-9466-4e2c0ca83cc9 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:09.006204 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1039/2074] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:09.008327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44538e57-0101-4c9f-bdfe-6db809bea61a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:09.029166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44538e57-0101-4c9f-bdfe-6db809bea61a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:09.074921 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0ce79-2cd1-42a0-9466-4e2c0ca83cc9 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:09.079873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923dde4-0105-4c03-9d82-5a3090fd673f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:09.080554 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923dde4-0105-4c03-9d82-5a3090fd673f admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:09.080839 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923dde4-0105-4c03-9d82-5a3090fd673f admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:09.081117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923dde4-0105-4c03-9d82-5a3090fd673f admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:09.081854 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923dde4-0105-4c03-9d82-5a3090fd673f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:09.083622 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923dde4-0105-4c03-9d82-5a3090fd673f admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:09.093533 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc0c9b-a492-4dcc-babb-a2040b8343fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:09.094222 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc0c9b-a492-4dcc-babb-a2040b8343fb admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:09.094516 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc0c9b-a492-4dcc-babb-a2040b8343fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:09.094912 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc0c9b-a492-4dcc-babb-a2040b8343fb admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:09.098529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1036/2075] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:08 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:35:09.098690 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc0c9b-a492-4dcc-babb-a2040b8343fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:09.098787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923dde4-0105-4c03-9d82-5a3090fd673f admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:09.098872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bc0c9b-a492-4dcc-babb-a2040b8343fb admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:09.100294 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-e923dde4-0105-4c03-9d82-5a3090fd673f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:09.115671 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:09.157721 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923dde4-0105-4c03-9d82-5a3090fd673f admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:09.161867 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37c8a3-b20c-4b9c-8920-ff9b5853ff20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:09.162668 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37c8a3-b20c-4b9c-8920-ff9b5853ff20 service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:09.163013 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37c8a3-b20c-4b9c-8920-ff9b5853ff20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:09.163477 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be37c8a3-b20c-4b9c-8920-ff9b5853ff20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:09.225508 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be37c8a3-b20c-4b9c-8920-ff9b5853ff20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:09.226943 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be37c8a3-b20c-4b9c-8920-ff9b5853ff20 service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:09.237085 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1037/2076] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:35:09.237925 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37c8a3-b20c-4b9c-8920-ff9b5853ff20 service cinder] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:09.248766 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be37c8a3-b20c-4b9c-8920-ff9b5853ff20 service cinder] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:09.255861 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a4585-6f13-479c-8431-c0197dfc8fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:09.255861 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a4585-6f13-479c-8431-c0197dfc8fe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:09.255861 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a4585-6f13-479c-8431-c0197dfc8fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:09.255861 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a4585-6f13-479c-8431-c0197dfc8fe2 admin admin] PATH_INFO: `/v3/projects/89f020ded21d40a7afe5cefad65714a0/users/7fc481224937454f8c7c5c8ea7e73031/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:09.275632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a4585-6f13-479c-8431-c0197dfc8fe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89f020ded21d40a7afe5cefad65714a0, user_id=7fc481224937454f8c7c5c8ea7e73031, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:09.278179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a4585-6f13-479c-8431-c0197dfc8fe2 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:09.291495 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1040/2077] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:35:09 2022] POST /identity/v3/users => generated 287 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:09.297935 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc0c9b-a492-4dcc-babb-a2040b8343fb admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:09.315398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc0c9b-a492-4dcc-babb-a2040b8343fb admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:09.317480 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:09.321696 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f955abd-d7b6-4208-be28-8f97c49afaed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:09.322760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f955abd-d7b6-4208-be28-8f97c49afaed admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:09.323145 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f955abd-d7b6-4208-be28-8f97c49afaed admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:09.323520 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f955abd-d7b6-4208-be28-8f97c49afaed admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:09.324962 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f955abd-d7b6-4208-be28-8f97c49afaed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:09.328163 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f955abd-d7b6-4208-be28-8f97c49afaed admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:09.349271 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1041/2078] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:35:09.353408 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f955abd-d7b6-4208-be28-8f97c49afaed admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:09.370563 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1038/2079] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:35:09 2022] PUT /identity/v3/projects/89f020ded21d40a7afe5cefad65714a0/users/7fc481224937454f8c7c5c8ea7e73031/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:09.374746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a4585-6f13-479c-8431-c0197dfc8fe2 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:09.379194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f955abd-d7b6-4208-be28-8f97c49afaed admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:09.469162 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a4585-6f13-479c-8431-c0197dfc8fe2 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:09.475668 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27accecd-2707-4689-ad44-9bf6fef09e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:09.476356 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf6ac5-a70a-40d2-8e46-2ab751155670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:09.476688 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27accecd-2707-4689-ad44-9bf6fef09e5b admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:09.477008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27accecd-2707-4689-ad44-9bf6fef09e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:09.477128 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf6ac5-a70a-40d2-8e46-2ab751155670 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:09.477289 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27accecd-2707-4689-ad44-9bf6fef09e5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:09.477399 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf6ac5-a70a-40d2-8e46-2ab751155670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:09.477606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bf6ac5-a70a-40d2-8e46-2ab751155670 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:09.478790 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27accecd-2707-4689-ad44-9bf6fef09e5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:09.478913 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf6ac5-a70a-40d2-8e46-2ab751155670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:09.480035 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27accecd-2707-4689-ad44-9bf6fef09e5b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:09.480160 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bf6ac5-a70a-40d2-8e46-2ab751155670 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:09.491319 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1039/2080] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:35:09.494401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1042/2081] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:09 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:35:09.495874 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf6ac5-a70a-40d2-8e46-2ab751155670 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:09.499218 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27accecd-2707-4689-ad44-9bf6fef09e5b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:09.509683 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bf6ac5-a70a-40d2-8e46-2ab751155670 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:09.516206 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27accecd-2707-4689-ad44-9bf6fef09e5b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:09.516718 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4a990d-2ba9-47e2-a18c-d9d66a8de500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:09.518035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a990d-2ba9-47e2-a18c-d9d66a8de500 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:09.518738 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a990d-2ba9-47e2-a18c-d9d66a8de500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:09.518963 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4a990d-2ba9-47e2-a18c-d9d66a8de500 admin admin] PATH_INFO: `/v3/projects/89f020ded21d40a7afe5cefad65714a0/users/7fc481224937454f8c7c5c8ea7e73031/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:09.521331 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e229ea-c5da-4d37-b554-c0df4883153d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:09.521753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4a990d-2ba9-47e2-a18c-d9d66a8de500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89f020ded21d40a7afe5cefad65714a0, user_id=7fc481224937454f8c7c5c8ea7e73031, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:09.522275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e229ea-c5da-4d37-b554-c0df4883153d admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:09.522558 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e229ea-c5da-4d37-b554-c0df4883153d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:09.522836 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e229ea-c5da-4d37-b554-c0df4883153d admin admin] PATH_INFO: `/v3/projects/7f663ab35e8749a6945b766b2f72f2a9/users/70df3efcdcf9425794f09f53cc0be942/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:09.526581 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4a990d-2ba9-47e2-a18c-d9d66a8de500 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:09.540823 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1040/2082] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:35:09 2022] PUT /identity/v3/projects/89f020ded21d40a7afe5cefad65714a0/users/7fc481224937454f8c7c5c8ea7e73031/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:09.541026 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e229ea-c5da-4d37-b554-c0df4883153d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f663ab35e8749a6945b766b2f72f2a9, user_id=70df3efcdcf9425794f09f53cc0be942, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:09.543876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e229ea-c5da-4d37-b554-c0df4883153d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:09.552337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d84d55-5a1b-4e4a-b4b0-2ce02a9e6647 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:09.552631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d84d55-5a1b-4e4a-b4b0-2ce02a9e6647 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:09.552916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d84d55-5a1b-4e4a-b4b0-2ce02a9e6647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:09.581308 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-c6d84d55-5a1b-4e4a-b4b0-2ce02a9e6647 None None] MFA Rules not processed for user `7fc481224937454f8c7c5c8ea7e73031`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:09.582575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-c6d84d55-5a1b-4e4a-b4b0-2ce02a9e6647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:09.651628 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1043/2083] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:35:09 2022] PUT /identity/v3/projects/7f663ab35e8749a6945b766b2f72f2a9/users/70df3efcdcf9425794f09f53cc0be942/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:09.663305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cbe6f7-4050-4c45-8391-5ccd278700c7 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:09.663305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cbe6f7-4050-4c45-8391-5ccd278700c7 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:09.663305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cbe6f7-4050-4c45-8391-5ccd278700c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:09.666937 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1041/2084] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:09 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:35:09.678156 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d84d55-5a1b-4e4a-b4b0-2ce02a9e6647 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:09.679662 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-c6d84d55-5a1b-4e4a-b4b0-2ce02a9e6647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:09.694528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-99cbe6f7-4050-4c45-8391-5ccd278700c7 None None] MFA Rules not processed for user `70df3efcdcf9425794f09f53cc0be942`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:09.695838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-99cbe6f7-4050-4c45-8391-5ccd278700c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:09.740700 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d84d55-5a1b-4e4a-b4b0-2ce02a9e6647 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:09.764581 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:09.776420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1044/2085] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:09.784282 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cbe6f7-4050-4c45-8391-5ccd278700c7 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:09.785725 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-99cbe6f7-4050-4c45-8391-5ccd278700c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:09.804064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838cafac-d6e9-4c09-9df5-4d6eb7e28278 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:09.805046 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cafac-d6e9-4c09-9df5-4d6eb7e28278 service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:09.805662 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cafac-d6e9-4c09-9df5-4d6eb7e28278 service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:09.805942 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cafac-d6e9-4c09-9df5-4d6eb7e28278 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:09.836141 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:09.875167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cbe6f7-4050-4c45-8391-5ccd278700c7 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:09.879364 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838cafac-d6e9-4c09-9df5-4d6eb7e28278 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:09.882326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838cafac-d6e9-4c09-9df5-4d6eb7e28278 service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:09.884117 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f3cd4-0696-454c-867e-8881c405b2dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:09.884881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f3cd4-0696-454c-867e-8881c405b2dc service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:09.885240 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f3cd4-0696-454c-867e-8881c405b2dc service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:09.885595 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f3cd4-0696-454c-867e-8881c405b2dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:09.894017 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f3cd4-0696-454c-867e-8881c405b2dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:09.895931 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1042/2086] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:09 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:35:09.896041 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f3cd4-0696-454c-867e-8881c405b2dc service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:09.910511 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1045/2087] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:35:10.245704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:10.283855 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838cafac-d6e9-4c09-9df5-4d6eb7e28278 service cinder] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:10.300639 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f3cd4-0696-454c-867e-8881c405b2dc service nova] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:10.313585 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f3cd4-0696-454c-867e-8881c405b2dc service nova] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:10.318497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7459b50-34da-4db5-adb4-33e59d2726aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:10.319169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7459b50-34da-4db5-adb4-33e59d2726aa service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:10.319497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7459b50-34da-4db5-adb4-33e59d2726aa service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:10.319960 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7459b50-34da-4db5-adb4-33e59d2726aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:10.323028 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e7459b50-34da-4db5-adb4-33e59d2726aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:10.365241 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838cafac-d6e9-4c09-9df5-4d6eb7e28278 service cinder] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:10.371227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19109f09-6a25-499e-9bc8-3f0d9db6be65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:10.372312 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19109f09-6a25-499e-9bc8-3f0d9db6be65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:10.372703 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19109f09-6a25-499e-9bc8-3f0d9db6be65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:10.373234 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19109f09-6a25-499e-9bc8-3f0d9db6be65 admin admin] PATH_INFO: `/v3/users/70df3efcdcf9425794f09f53cc0be942` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:10.374812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19109f09-6a25-499e-9bc8-3f0d9db6be65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70df3efcdcf9425794f09f53cc0be942)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:10.377131 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19109f09-6a25-499e-9bc8-3f0d9db6be65 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:10.405776 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7459b50-34da-4db5-adb4-33e59d2726aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:10.407674 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7459b50-34da-4db5-adb4-33e59d2726aa service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:10.419837 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1046/2088] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:10 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:35:10.436816 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:10.599159 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-19109f09-6a25-499e-9bc8-3f0d9db6be65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70df3efcdcf9425794f09f53cc0be942'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:10.681392 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-19109f09-6a25-499e-9bc8-3f0d9db6be65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70df3efcdcf9425794f09f53cc0be942'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:10.697989 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-19109f09-6a25-499e-9bc8-3f0d9db6be65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70df3efcdcf9425794f09f53cc0be942'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:10.698854 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-19109f09-6a25-499e-9bc8-3f0d9db6be65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70df3efcdcf9425794f09f53cc0be942'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:10.699540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-19109f09-6a25-499e-9bc8-3f0d9db6be65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70df3efcdcf9425794f09f53cc0be942'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:10.715630 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1043/2089] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:35:10 2022] DELETE /identity/v3/users/70df3efcdcf9425794f09f53cc0be942 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:10.717743 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813d195-00be-40d6-ad41-0f28db10b708 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:10.718210 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813d195-00be-40d6-ad41-0f28db10b708 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:10.718489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8813d195-00be-40d6-ad41-0f28db10b708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:10.737964 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8813d195-00be-40d6-ad41-0f28db10b708 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:10.739234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8813d195-00be-40d6-ad41-0f28db10b708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:10.804166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1047/2090] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:10.814276 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19109f09-6a25-499e-9bc8-3f0d9db6be65 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:10.816082 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-19109f09-6a25-499e-9bc8-3f0d9db6be65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:10.880912 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19109f09-6a25-499e-9bc8-3f0d9db6be65 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:10.885916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1907523-24aa-4776-9797-2b6a1cd0aaf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:10.901691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1907523-24aa-4776-9797-2b6a1cd0aaf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:10.901691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1907523-24aa-4776-9797-2b6a1cd0aaf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:10.901691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1907523-24aa-4776-9797-2b6a1cd0aaf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:10.911972 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1907523-24aa-4776-9797-2b6a1cd0aaf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:10.913484 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1907523-24aa-4776-9797-2b6a1cd0aaf8 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:10.927864 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1044/2091] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:35:11.107624 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8813d195-00be-40d6-ad41-0f28db10b708 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:11.109031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8813d195-00be-40d6-ad41-0f28db10b708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:11.121423 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8813d195-00be-40d6-ad41-0f28db10b708 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:11.129596 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed7f52-2e4d-443c-8644-49e67c5aa1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:11.130417 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed7f52-2e4d-443c-8644-49e67c5aa1e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:11.130701 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed7f52-2e4d-443c-8644-49e67c5aa1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:11.130975 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ed7f52-2e4d-443c-8644-49e67c5aa1e5 admin admin] PATH_INFO: `/v3/users/421aad7e0ebc4d1abac74e91c145545b` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:11.132262 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ed7f52-2e4d-443c-8644-49e67c5aa1e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=421aad7e0ebc4d1abac74e91c145545b)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:11.134432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ed7f52-2e4d-443c-8644-49e67c5aa1e5 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:11.405092 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a7ed7f52-2e4d-443c-8644-49e67c5aa1e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '421aad7e0ebc4d1abac74e91c145545b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:11.405831 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a7ed7f52-2e4d-443c-8644-49e67c5aa1e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '421aad7e0ebc4d1abac74e91c145545b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:11.413051 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a7ed7f52-2e4d-443c-8644-49e67c5aa1e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '421aad7e0ebc4d1abac74e91c145545b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:11.413749 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a7ed7f52-2e4d-443c-8644-49e67c5aa1e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '421aad7e0ebc4d1abac74e91c145545b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:11.475380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a7ed7f52-2e4d-443c-8644-49e67c5aa1e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '421aad7e0ebc4d1abac74e91c145545b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:11.496811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1048/2092] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:35:11 2022] DELETE /identity/v3/users/421aad7e0ebc4d1abac74e91c145545b => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:11.570253 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1907523-24aa-4776-9797-2b6a1cd0aaf8 service neutron] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:11.572387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-e1907523-24aa-4776-9797-2b6a1cd0aaf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:11.588413 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1907523-24aa-4776-9797-2b6a1cd0aaf8 service neutron] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:11.606748 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:11.671103 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55bfbda-db65-4ec0-8010-4fc541973305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:11.672308 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55bfbda-db65-4ec0-8010-4fc541973305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:11.672308 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55bfbda-db65-4ec0-8010-4fc541973305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:11.672623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55bfbda-db65-4ec0-8010-4fc541973305 admin admin] PATH_INFO: `/v3/projects/7f663ab35e8749a6945b766b2f72f2a9` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:11.673777 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55bfbda-db65-4ec0-8010-4fc541973305 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f663ab35e8749a6945b766b2f72f2a9)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:11.675738 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55bfbda-db65-4ec0-8010-4fc541973305 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:11.826376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a55bfbda-db65-4ec0-8010-4fc541973305 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f663ab35e8749a6945b766b2f72f2a9', 'request_id': 'req-a55bfbda-db65-4ec0-8010-4fc541973305'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:11.832578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a55bfbda-db65-4ec0-8010-4fc541973305 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f663ab35e8749a6945b766b2f72f2a9', 'request_id': 'req-a55bfbda-db65-4ec0-8010-4fc541973305'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:11.838898 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a55bfbda-db65-4ec0-8010-4fc541973305 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f663ab35e8749a6945b766b2f72f2a9', 'request_id': 'req-a55bfbda-db65-4ec0-8010-4fc541973305'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:11.848883 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed7f52-2e4d-443c-8644-49e67c5aa1e5 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:11.850386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a7ed7f52-2e4d-443c-8644-49e67c5aa1e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:11.868896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ed7f52-2e4d-443c-8644-49e67c5aa1e5 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:11.923314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:11.933951 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec02b8-e1d0-4ab8-a19d-0c6a856a9784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:11.934819 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec02b8-e1d0-4ab8-a19d-0c6a856a9784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:11.935108 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec02b8-e1d0-4ab8-a19d-0c6a856a9784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:11.935416 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec02b8-e1d0-4ab8-a19d-0c6a856a9784 admin admin] PATH_INFO: `/v3/projects/70054d80fd034da68fab86c5145ffb1c` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:11.936595 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec02b8-e1d0-4ab8-a19d-0c6a856a9784 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70054d80fd034da68fab86c5145ffb1c)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:11.938669 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec02b8-e1d0-4ab8-a19d-0c6a856a9784 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:11.962423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a55bfbda-db65-4ec0-8010-4fc541973305 admin admin] The token cache is being invalidate because project 7f663ab35e8749a6945b766b2f72f2a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:35:11.963302 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a55bfbda-db65-4ec0-8010-4fc541973305 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:11.964060 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a55bfbda-db65-4ec0-8010-4fc541973305 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:11.965776 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1045/2093] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:35:11 2022] DELETE /identity/v3/projects/7f663ab35e8749a6945b766b2f72f2a9 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:11.974472 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759515f-3007-4fb5-b904-eecc888cdda7 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:11.974827 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759515f-3007-4fb5-b904-eecc888cdda7 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:11.975111 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c759515f-3007-4fb5-b904-eecc888cdda7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:11.996331 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-c759515f-3007-4fb5-b904-eecc888cdda7 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:11.997994 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-c759515f-3007-4fb5-b904-eecc888cdda7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:12.006801 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1046/2094] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:35:12.014931 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897acb9-6a2f-44cd-a442-7b5caa9008b7 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:12.015221 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897acb9-6a2f-44cd-a442-7b5caa9008b7 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:12.015534 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5897acb9-6a2f-44cd-a442-7b5caa9008b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:12.033600 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-5897acb9-6a2f-44cd-a442-7b5caa9008b7 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:12.034837 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-5897acb9-6a2f-44cd-a442-7b5caa9008b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:12.042949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1047/2095] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:35:12.048677 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897acb9-6a2f-44cd-a442-7b5caa9008b7 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:12.058612 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5897acb9-6a2f-44cd-a442-7b5caa9008b7 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:12.063085 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373467a-e134-40e3-b8b9-859b6dd9debe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:12.063802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373467a-e134-40e3-b8b9-859b6dd9debe admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:12.064089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373467a-e134-40e3-b8b9-859b6dd9debe admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:12.064367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c373467a-e134-40e3-b8b9-859b6dd9debe admin admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:12.065114 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c373467a-e134-40e3-b8b9-859b6dd9debe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:35:12.065411 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c373467a-e134-40e3-b8b9-859b6dd9debe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:12.067041 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c373467a-e134-40e3-b8b9-859b6dd9debe admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:12.083039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1048/2096] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:35:12.084679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373467a-e134-40e3-b8b9-859b6dd9debe admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:12.088369 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-57ec02b8-e1d0-4ab8-a19d-0c6a856a9784 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70054d80fd034da68fab86c5145ffb1c', 'request_id': 'req-57ec02b8-e1d0-4ab8-a19d-0c6a856a9784'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:12.094870 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c373467a-e134-40e3-b8b9-859b6dd9debe admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:12.151208 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62227fbe-53bb-476c-8665-dd4064167131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:12.151905 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62227fbe-53bb-476c-8665-dd4064167131 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:12.152190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62227fbe-53bb-476c-8665-dd4064167131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:12.152467 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62227fbe-53bb-476c-8665-dd4064167131 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:12.153161 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62227fbe-53bb-476c-8665-dd4064167131 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:12.155369 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62227fbe-53bb-476c-8665-dd4064167131 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:12.213981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-57ec02b8-e1d0-4ab8-a19d-0c6a856a9784 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70054d80fd034da68fab86c5145ffb1c', 'request_id': 'req-57ec02b8-e1d0-4ab8-a19d-0c6a856a9784'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:12.220022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-57ec02b8-e1d0-4ab8-a19d-0c6a856a9784 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70054d80fd034da68fab86c5145ffb1c', 'request_id': 'req-57ec02b8-e1d0-4ab8-a19d-0c6a856a9784'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:12.232563 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-57ec02b8-e1d0-4ab8-a19d-0c6a856a9784 admin admin] The token cache is being invalidate because project 70054d80fd034da68fab86c5145ffb1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:35:12.232855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-57ec02b8-e1d0-4ab8-a19d-0c6a856a9784 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:12.233517 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-57ec02b8-e1d0-4ab8-a19d-0c6a856a9784 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:12.235486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1049/2097] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:35:11 2022] DELETE /identity/v3/projects/70054d80fd034da68fab86c5145ffb1c => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:35:12.239178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec02b8-e1d0-4ab8-a19d-0c6a856a9784 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:12.242230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-57ec02b8-e1d0-4ab8-a19d-0c6a856a9784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:12.254443 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1049/2098] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:35:12 2022] POST /identity/v3/projects => generated 385 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:35:12.259525 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62227fbe-53bb-476c-8665-dd4064167131 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:12.261661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-62227fbe-53bb-476c-8665-dd4064167131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:12.263329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec02b8-e1d0-4ab8-a19d-0c6a856a9784 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:12.330060 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d83d943-8e14-4f90-9fea-1dfe9c0129b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:12.330366 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62227fbe-53bb-476c-8665-dd4064167131 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:12.331473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83d943-8e14-4f90-9fea-1dfe9c0129b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:12.331892 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83d943-8e14-4f90-9fea-1dfe9c0129b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:12.332299 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d83d943-8e14-4f90-9fea-1dfe9c0129b6 admin admin] PATH_INFO: `/v3/users/296f7b3dab274b1083efe4105959a15c` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:12.333772 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83d943-8e14-4f90-9fea-1dfe9c0129b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=296f7b3dab274b1083efe4105959a15c)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:12.335926 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10eeafd-f472-4d09-8646-a2a37d8e4e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:12.336220 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d83d943-8e14-4f90-9fea-1dfe9c0129b6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:12.337170 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10eeafd-f472-4d09-8646-a2a37d8e4e72 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:12.337455 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10eeafd-f472-4d09-8646-a2a37d8e4e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:12.337731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10eeafd-f472-4d09-8646-a2a37d8e4e72 admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:12.338429 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10eeafd-f472-4d09-8646-a2a37d8e4e72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:12.340454 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10eeafd-f472-4d09-8646-a2a37d8e4e72 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:12.470639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1050/2099] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:35:12 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:12.474778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10eeafd-f472-4d09-8646-a2a37d8e4e72 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:12.485594 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10eeafd-f472-4d09-8646-a2a37d8e4e72 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:12.490722 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116672a9-47f3-4c97-b01c-ac5cdf61e4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:12.491423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116672a9-47f3-4c97-b01c-ac5cdf61e4d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:12.491706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116672a9-47f3-4c97-b01c-ac5cdf61e4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:12.491985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116672a9-47f3-4c97-b01c-ac5cdf61e4d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:12.492911 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116672a9-47f3-4c97-b01c-ac5cdf61e4d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:12.495107 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116672a9-47f3-4c97-b01c-ac5cdf61e4d6 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:12.512647 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1051/2100] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:35:12.514658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116672a9-47f3-4c97-b01c-ac5cdf61e4d6 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:12.530227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116672a9-47f3-4c97-b01c-ac5cdf61e4d6 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:12.530493 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a5cee-2819-43f9-8f14-4a41bad36563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:12.531169 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a5cee-2819-43f9-8f14-4a41bad36563 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:12.531488 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a5cee-2819-43f9-8f14-4a41bad36563 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:12.531765 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a5cee-2819-43f9-8f14-4a41bad36563 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:12.532493 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5a5cee-2819-43f9-8f14-4a41bad36563 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:12.534085 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5a5cee-2819-43f9-8f14-4a41bad36563 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:12.545643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1052/2101] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:35:12.549707 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a5cee-2819-43f9-8f14-4a41bad36563 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:12.560100 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a5cee-2819-43f9-8f14-4a41bad36563 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:12.564708 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1078c-8f40-48e0-9f8d-0e8268ce16c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:12.565573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1078c-8f40-48e0-9f8d-0e8268ce16c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:12.565862 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1078c-8f40-48e0-9f8d-0e8268ce16c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:12.566197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1078c-8f40-48e0-9f8d-0e8268ce16c5 admin admin] PATH_INFO: `/v3/projects/ecc2f6432157479e8115fcea39f90cae/users/bf5b88ea871b4c7687576406fede7c31/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:12.585460 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe1078c-8f40-48e0-9f8d-0e8268ce16c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecc2f6432157479e8115fcea39f90cae, user_id=bf5b88ea871b4c7687576406fede7c31, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:12.585460 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe1078c-8f40-48e0-9f8d-0e8268ce16c5 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:12.638322 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4d83d943-8e14-4f90-9fea-1dfe9c0129b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '296f7b3dab274b1083efe4105959a15c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:12.639021 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4d83d943-8e14-4f90-9fea-1dfe9c0129b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '296f7b3dab274b1083efe4105959a15c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:12.646003 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4d83d943-8e14-4f90-9fea-1dfe9c0129b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '296f7b3dab274b1083efe4105959a15c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:12.646655 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4d83d943-8e14-4f90-9fea-1dfe9c0129b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '296f7b3dab274b1083efe4105959a15c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:12.656517 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1053/2102] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:35:12 2022] PUT /identity/v3/projects/ecc2f6432157479e8115fcea39f90cae/users/bf5b88ea871b4c7687576406fede7c31/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:35:12.657478 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1078c-8f40-48e0-9f8d-0e8268ce16c5 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:12.665595 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-1fe1078c-8f40-48e0-9f8d-0e8268ce16c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:12.710125 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4d83d943-8e14-4f90-9fea-1dfe9c0129b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '296f7b3dab274b1083efe4105959a15c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:12.733621 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1050/2103] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:35:12 2022] DELETE /identity/v3/users/296f7b3dab274b1083efe4105959a15c => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:35:12.733621 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69e3eb-5f21-45b7-8e59-5b74c78598b8 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:12.733621 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69e3eb-5f21-45b7-8e59-5b74c78598b8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:12.733621 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd69e3eb-5f21-45b7-8e59-5b74c78598b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:12.743634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1078c-8f40-48e0-9f8d-0e8268ce16c5 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:12.767333 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-dd69e3eb-5f21-45b7-8e59-5b74c78598b8 None None] MFA Rules not processed for user `bf5b88ea871b4c7687576406fede7c31`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:12.769008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-dd69e3eb-5f21-45b7-8e59-5b74c78598b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:12.790994 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:12.809616 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74afd8d-11d0-439e-826a-8e22d0070292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:12.810739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74afd8d-11d0-439e-826a-8e22d0070292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:12.811165 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74afd8d-11d0-439e-826a-8e22d0070292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:12.811688 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74afd8d-11d0-439e-826a-8e22d0070292 admin admin] PATH_INFO: `/v3/users/e9aac5f2e6a7416fb49608f0bd21c0a2` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:12.814063 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74afd8d-11d0-439e-826a-8e22d0070292 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9aac5f2e6a7416fb49608f0bd21c0a2)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:12.816683 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74afd8d-11d0-439e-826a-8e22d0070292 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:12.849869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1051/2104] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:12.854862 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69e3eb-5f21-45b7-8e59-5b74c78598b8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:12.856226 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-dd69e3eb-5f21-45b7-8e59-5b74c78598b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:12.869694 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd69e3eb-5f21-45b7-8e59-5b74c78598b8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:12.874495 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b66d3-c906-41ff-abeb-cb7cf6955a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:12.875159 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b66d3-c906-41ff-abeb-cb7cf6955a5c admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:12.875474 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b66d3-c906-41ff-abeb-cb7cf6955a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:12.875754 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b66d3-c906-41ff-abeb-cb7cf6955a5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:12.876697 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9b66d3-c906-41ff-abeb-cb7cf6955a5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:12.878395 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9b66d3-c906-41ff-abeb-cb7cf6955a5c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:12.964833 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:12.972574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1052/2105] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:35:12 2022] POST /identity/v3/projects => generated 385 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:12.976366 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b66d3-c906-41ff-abeb-cb7cf6955a5c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:13.068330 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b66d3-c906-41ff-abeb-cb7cf6955a5c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:13.073765 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-f74afd8d-11d0-439e-826a-8e22d0070292 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9aac5f2e6a7416fb49608f0bd21c0a2'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:13.074066 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3be88-f224-460c-ade1-06f4c950cdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:13.074739 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3be88-f224-460c-ade1-06f4c950cdd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:13.075118 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3be88-f224-460c-ade1-06f4c950cdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:13.075426 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3be88-f224-460c-ade1-06f4c950cdd3 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:13.076118 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3be88-f224-460c-ade1-06f4c950cdd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:13.078217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3be88-f224-460c-ade1-06f4c950cdd3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:13.161509 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-f74afd8d-11d0-439e-826a-8e22d0070292 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9aac5f2e6a7416fb49608f0bd21c0a2'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:13.174116 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-f74afd8d-11d0-439e-826a-8e22d0070292 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9aac5f2e6a7416fb49608f0bd21c0a2'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:13.174842 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-f74afd8d-11d0-439e-826a-8e22d0070292 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9aac5f2e6a7416fb49608f0bd21c0a2'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:13.175532 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-f74afd8d-11d0-439e-826a-8e22d0070292 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9aac5f2e6a7416fb49608f0bd21c0a2'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:13.187715 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1054/2106] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:35:12 2022] DELETE /identity/v3/users/e9aac5f2e6a7416fb49608f0bd21c0a2 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:13.188647 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74afd8d-11d0-439e-826a-8e22d0070292 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:13.189928 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-f74afd8d-11d0-439e-826a-8e22d0070292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:13.198184 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1053/2107] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:35:12 2022] POST /identity/v3/users => generated 299 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:35:13.198368 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783c35f-1619-430f-85af-4038ad387011 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:13.198664 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783c35f-1619-430f-85af-4038ad387011 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:13.198889 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f783c35f-1619-430f-85af-4038ad387011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:13.216759 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f783c35f-1619-430f-85af-4038ad387011 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:13.218009 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f783c35f-1619-430f-85af-4038ad387011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:13.263568 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74afd8d-11d0-439e-826a-8e22d0070292 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:13.268990 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eff180-4b25-4c01-9e64-b8d74808600c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:13.269853 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eff180-4b25-4c01-9e64-b8d74808600c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:13.270160 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eff180-4b25-4c01-9e64-b8d74808600c admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:13.270557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eff180-4b25-4c01-9e64-b8d74808600c admin admin] PATH_INFO: `/v3/projects/b2199044d60d465fa6d564275014ca4c` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:13.271708 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eff180-4b25-4c01-9e64-b8d74808600c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2199044d60d465fa6d564275014ca4c)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:13.273628 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eff180-4b25-4c01-9e64-b8d74808600c admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:13.286983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1054/2108] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:13.287889 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f783c35f-1619-430f-85af-4038ad387011 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:13.289150 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f783c35f-1619-430f-85af-4038ad387011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:13.300734 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f783c35f-1619-430f-85af-4038ad387011 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:13.307999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba31bb-70bc-4845-bcc6-581bbb1a4d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:13.308680 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba31bb-70bc-4845-bcc6-581bbb1a4d64 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:13.308965 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba31bb-70bc-4845-bcc6-581bbb1a4d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:13.309239 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aba31bb-70bc-4845-bcc6-581bbb1a4d64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:13.309971 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba31bb-70bc-4845-bcc6-581bbb1a4d64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:13.311707 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aba31bb-70bc-4845-bcc6-581bbb1a4d64 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:13.322682 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1055/2109] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:35:13.323504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba31bb-70bc-4845-bcc6-581bbb1a4d64 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:13.325121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2aba31bb-70bc-4845-bcc6-581bbb1a4d64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:13.381128 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aba31bb-70bc-4845-bcc6-581bbb1a4d64 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:13.392164 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-46eff180-4b25-4c01-9e64-b8d74808600c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2199044d60d465fa6d564275014ca4c', 'request_id': 'req-46eff180-4b25-4c01-9e64-b8d74808600c'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:13.399179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-46eff180-4b25-4c01-9e64-b8d74808600c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2199044d60d465fa6d564275014ca4c', 'request_id': 'req-46eff180-4b25-4c01-9e64-b8d74808600c'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:13.405478 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-46eff180-4b25-4c01-9e64-b8d74808600c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2199044d60d465fa6d564275014ca4c', 'request_id': 'req-46eff180-4b25-4c01-9e64-b8d74808600c'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:13.431155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38db471-6c9e-4b5b-9689-835e9f82d0f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:13.431939 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38db471-6c9e-4b5b-9689-835e9f82d0f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:13.432233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38db471-6c9e-4b5b-9689-835e9f82d0f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:13.432513 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38db471-6c9e-4b5b-9689-835e9f82d0f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:13.491909 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38db471-6c9e-4b5b-9689-835e9f82d0f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:13.493918 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38db471-6c9e-4b5b-9689-835e9f82d0f5 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:13.503898 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1056/2110] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:35:13.504721 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38db471-6c9e-4b5b-9689-835e9f82d0f5 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:13.514764 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38db471-6c9e-4b5b-9689-835e9f82d0f5 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:13.519665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ce9d9-47ff-4f44-8874-d6f8f057a256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:13.520608 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ce9d9-47ff-4f44-8874-d6f8f057a256 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:13.521258 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ce9d9-47ff-4f44-8874-d6f8f057a256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:13.521258 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7ce9d9-47ff-4f44-8874-d6f8f057a256 admin admin] PATH_INFO: `/v3/projects/6baf268f62fe4df0861aea40f9ec8be4/users/353a8bed680d411886462e32483dcaee/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:13.527363 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-46eff180-4b25-4c01-9e64-b8d74808600c admin admin] The token cache is being invalidate because project b2199044d60d465fa6d564275014ca4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:35:13.527661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-46eff180-4b25-4c01-9e64-b8d74808600c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:13.528330 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-46eff180-4b25-4c01-9e64-b8d74808600c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:13.530401 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1055/2111] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:35:13 2022] DELETE /identity/v3/projects/b2199044d60d465fa6d564275014ca4c => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:13.538965 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6e15a-1f9a-4acd-bb67-70b07d9f506d None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:13.539627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6e15a-1f9a-4acd-bb67-70b07d9f506d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:13.540106 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ce9d9-47ff-4f44-8874-d6f8f057a256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6baf268f62fe4df0861aea40f9ec8be4, user_id=353a8bed680d411886462e32483dcaee, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:13.540455 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b6e15a-1f9a-4acd-bb67-70b07d9f506d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:13.546408 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7ce9d9-47ff-4f44-8874-d6f8f057a256 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:13.564049 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-17b6e15a-1f9a-4acd-bb67-70b07d9f506d None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:13.564785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-17b6e15a-1f9a-4acd-bb67-70b07d9f506d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:13.574899 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1056/2112] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:35:13.583850 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496d4a5-779b-4adf-83a6-900b5e0d6191 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:13.584420 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496d4a5-779b-4adf-83a6-900b5e0d6191 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:13.584546 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e496d4a5-779b-4adf-83a6-900b5e0d6191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:13.728385 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-e496d4a5-779b-4adf-83a6-900b5e0d6191 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:13.728385 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-e496d4a5-779b-4adf-83a6-900b5e0d6191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:13.728385 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1057/2113] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:35:13.728385 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496d4a5-779b-4adf-83a6-900b5e0d6191 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:13.730361 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e496d4a5-779b-4adf-83a6-900b5e0d6191 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:13.735079 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b084cc-d4db-4417-b3b1-e5586a369a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:13.735768 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b084cc-d4db-4417-b3b1-e5586a369a91 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:13.736060 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b084cc-d4db-4417-b3b1-e5586a369a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:13.736337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b084cc-d4db-4417-b3b1-e5586a369a91 admin admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:13.737065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b084cc-d4db-4417-b3b1-e5586a369a91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:35:13.737372 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b084cc-d4db-4417-b3b1-e5586a369a91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:13.738991 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b084cc-d4db-4417-b3b1-e5586a369a91 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:13.751230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1058/2114] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:35:13.755255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b084cc-d4db-4417-b3b1-e5586a369a91 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:13.765362 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b084cc-d4db-4417-b3b1-e5586a369a91 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:13.771117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ca9a4-b4e7-402b-be6b-14f686c1bab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:13.772053 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ca9a4-b4e7-402b-be6b-14f686c1bab1 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:13.772339 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ca9a4-b4e7-402b-be6b-14f686c1bab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:13.772615 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ca9a4-b4e7-402b-be6b-14f686c1bab1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:13.773307 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ca9a4-b4e7-402b-be6b-14f686c1bab1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:13.775352 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ca9a4-b4e7-402b-be6b-14f686c1bab1 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:13.985608 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1057/2115] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:35:13 2022] PUT /identity/v3/projects/6baf268f62fe4df0861aea40f9ec8be4/users/353a8bed680d411886462e32483dcaee/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:13.985608 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ce9d9-47ff-4f44-8874-d6f8f057a256 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:13.986214 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1059/2116] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:35:13 2022] POST /identity/v3/projects => generated 383 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:35:13.986214 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ca9a4-b4e7-402b-be6b-14f686c1bab1 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:13.986618 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ae7ce9d9-47ff-4f44-8874-d6f8f057a256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:14.060656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:14.096343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ca9a4-b4e7-402b-be6b-14f686c1bab1 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:14.098746 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7ce9d9-47ff-4f44-8874-d6f8f057a256 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:14.103022 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432adb24-bcfe-47cc-91b2-6408f9e86d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:14.104105 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec51892-29f0-40a8-894c-ba17f8cc7d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:14.104782 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec51892-29f0-40a8-894c-ba17f8cc7d39 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:14.105020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432adb24-bcfe-47cc-91b2-6408f9e86d21 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:14.105131 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec51892-29f0-40a8-894c-ba17f8cc7d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:14.105334 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec51892-29f0-40a8-894c-ba17f8cc7d39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:14.105764 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432adb24-bcfe-47cc-91b2-6408f9e86d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:14.106075 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec51892-29f0-40a8-894c-ba17f8cc7d39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:14.106392 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432adb24-bcfe-47cc-91b2-6408f9e86d21 admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:14.107738 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec51892-29f0-40a8-894c-ba17f8cc7d39 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:14.107953 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432adb24-bcfe-47cc-91b2-6408f9e86d21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:14.110195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432adb24-bcfe-47cc-91b2-6408f9e86d21 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:14.118322 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1058/2117] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:35:14.122201 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec51892-29f0-40a8-894c-ba17f8cc7d39 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:14.132426 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:14.136295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec51892-29f0-40a8-894c-ba17f8cc7d39 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:14.141014 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4915a8-c366-47e4-8c28-000954742084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:14.142053 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4915a8-c366-47e4-8c28-000954742084 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:14.142432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4915a8-c366-47e4-8c28-000954742084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:14.142752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4915a8-c366-47e4-8c28-000954742084 admin admin] PATH_INFO: `/v3/projects/6baf268f62fe4df0861aea40f9ec8be4/users/353a8bed680d411886462e32483dcaee/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:14.144915 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4915a8-c366-47e4-8c28-000954742084 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6baf268f62fe4df0861aea40f9ec8be4, user_id=353a8bed680d411886462e32483dcaee, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:14.147651 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4915a8-c366-47e4-8c28-000954742084 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:14.158984 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1059/2118] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:35:14 2022] PUT /identity/v3/projects/6baf268f62fe4df0861aea40f9ec8be4/users/353a8bed680d411886462e32483dcaee/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:35:14.169560 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f48d7-2951-4851-8d2e-8c9d527b6b7d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:14.169849 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f48d7-2951-4851-8d2e-8c9d527b6b7d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:14.170128 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97f48d7-2951-4851-8d2e-8c9d527b6b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:14.198577 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f97f48d7-2951-4851-8d2e-8c9d527b6b7d None None] MFA Rules not processed for user `353a8bed680d411886462e32483dcaee`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:14.199833 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f97f48d7-2951-4851-8d2e-8c9d527b6b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:14.255048 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1060/2119] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:35:13 2022] POST /identity/v3/users => generated 292 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:14.258690 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432adb24-bcfe-47cc-91b2-6408f9e86d21 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:14.261086 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-432adb24-bcfe-47cc-91b2-6408f9e86d21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:14.274053 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432adb24-bcfe-47cc-91b2-6408f9e86d21 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:14.285763 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1060/2120] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:35:14.286698 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f48d7-2951-4851-8d2e-8c9d527b6b7d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:14.338344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0ebf4-3740-4fe3-87f9-4be4cd3380ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:14.339530 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0ebf4-3740-4fe3-87f9-4be4cd3380ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:14.339816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0ebf4-3740-4fe3-87f9-4be4cd3380ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:14.340095 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c0ebf4-3740-4fe3-87f9-4be4cd3380ec admin admin] PATH_INFO: `/v3/users/23c6c70a0cb84dc4903c042afb441e2a` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:14.350855 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0ebf4-3740-4fe3-87f9-4be4cd3380ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=23c6c70a0cb84dc4903c042afb441e2a)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:14.350855 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c0ebf4-3740-4fe3-87f9-4be4cd3380ec admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:14.363469 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97f48d7-2951-4851-8d2e-8c9d527b6b7d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:14.369143 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a65f8-f2a6-4e67-bb1e-1c891722ba31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:14.369823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a65f8-f2a6-4e67-bb1e-1c891722ba31 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:14.370111 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a65f8-f2a6-4e67-bb1e-1c891722ba31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:14.370392 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a65f8-f2a6-4e67-bb1e-1c891722ba31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:14.371137 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a65f8-f2a6-4e67-bb1e-1c891722ba31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:14.375678 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5a65f8-f2a6-4e67-bb1e-1c891722ba31 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:14.389952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1061/2121] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:35:14.389952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a65f8-f2a6-4e67-bb1e-1c891722ba31 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:14.390487 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2d5a65f8-f2a6-4e67-bb1e-1c891722ba31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:14.448178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5a65f8-f2a6-4e67-bb1e-1c891722ba31 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:14.451802 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac09e9-0eae-4818-babf-91f7417c7681 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:14.452446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac09e9-0eae-4818-babf-91f7417c7681 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:14.452733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac09e9-0eae-4818-babf-91f7417c7681 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:14.453015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ac09e9-0eae-4818-babf-91f7417c7681 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:14.461896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac09e9-0eae-4818-babf-91f7417c7681 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:14.464008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ac09e9-0eae-4818-babf-91f7417c7681 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:14.473261 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1062/2122] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:35:14.474039 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac09e9-0eae-4818-babf-91f7417c7681 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:14.475343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-26ac09e9-0eae-4818-babf-91f7417c7681 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:14.488991 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ac09e9-0eae-4818-babf-91f7417c7681 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:14.493712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe5f18d-62d9-439a-ad41-c14fdbf0dfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:14.494619 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5f18d-62d9-439a-ad41-c14fdbf0dfe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:14.494904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5f18d-62d9-439a-ad41-c14fdbf0dfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:14.495182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe5f18d-62d9-439a-ad41-c14fdbf0dfe1 admin admin] PATH_INFO: `/v3/projects/9f0377219b474143b09431f000d38f28` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:14.496395 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe5f18d-62d9-439a-ad41-c14fdbf0dfe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f0377219b474143b09431f000d38f28)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:14.498570 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe5f18d-62d9-439a-ad41-c14fdbf0dfe1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:14.544908 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:14.549151 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d6c0ebf4-3740-4fe3-87f9-4be4cd3380ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23c6c70a0cb84dc4903c042afb441e2a'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:14.595927 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6fe5f18d-62d9-439a-ad41-c14fdbf0dfe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f0377219b474143b09431f000d38f28', 'request_id': 'req-6fe5f18d-62d9-439a-ad41-c14fdbf0dfe1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:14.624398 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d6c0ebf4-3740-4fe3-87f9-4be4cd3380ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23c6c70a0cb84dc4903c042afb441e2a'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:14.624865 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:14.637535 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d6c0ebf4-3740-4fe3-87f9-4be4cd3380ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23c6c70a0cb84dc4903c042afb441e2a'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:14.638178 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d6c0ebf4-3740-4fe3-87f9-4be4cd3380ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23c6c70a0cb84dc4903c042afb441e2a'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:14.638775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d6c0ebf4-3740-4fe3-87f9-4be4cd3380ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23c6c70a0cb84dc4903c042afb441e2a'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:14.651592 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1061/2123] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:35:14 2022] DELETE /identity/v3/users/23c6c70a0cb84dc4903c042afb441e2a => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:14.670429 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0ebf4-3740-4fe3-87f9-4be4cd3380ec admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:14.670429 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d6c0ebf4-3740-4fe3-87f9-4be4cd3380ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:14.715907 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c0ebf4-3740-4fe3-87f9-4be4cd3380ec admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:14.720497 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a351e4b-b006-41c1-8fbb-9f90a424cf81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:14.721154 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a351e4b-b006-41c1-8fbb-9f90a424cf81 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:14.721440 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a351e4b-b006-41c1-8fbb-9f90a424cf81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:14.721719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a351e4b-b006-41c1-8fbb-9f90a424cf81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:14.722441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a351e4b-b006-41c1-8fbb-9f90a424cf81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:14.724075 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a351e4b-b006-41c1-8fbb-9f90a424cf81 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:14.734076 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1062/2124] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:35:14.734861 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a351e4b-b006-41c1-8fbb-9f90a424cf81 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:14.736453 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-5a351e4b-b006-41c1-8fbb-9f90a424cf81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:14.756317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a351e4b-b006-41c1-8fbb-9f90a424cf81 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:14.756317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe421b8-8c05-4e12-98d5-ac075d13610a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:14.756317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe421b8-8c05-4e12-98d5-ac075d13610a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:14.756317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe421b8-8c05-4e12-98d5-ac075d13610a admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:14.756733 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe421b8-8c05-4e12-98d5-ac075d13610a admin admin] PATH_INFO: `/v3/users/bf5b88ea871b4c7687576406fede7c31` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:14.757338 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe421b8-8c05-4e12-98d5-ac075d13610a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf5b88ea871b4c7687576406fede7c31)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:14.759783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe421b8-8c05-4e12-98d5-ac075d13610a admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:14.784826 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6fe5f18d-62d9-439a-ad41-c14fdbf0dfe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f0377219b474143b09431f000d38f28', 'request_id': 'req-6fe5f18d-62d9-439a-ad41-c14fdbf0dfe1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:14.791119 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6fe5f18d-62d9-439a-ad41-c14fdbf0dfe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f0377219b474143b09431f000d38f28', 'request_id': 'req-6fe5f18d-62d9-439a-ad41-c14fdbf0dfe1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:14.802079 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6fe5f18d-62d9-439a-ad41-c14fdbf0dfe1 admin admin] The token cache is being invalidate because project 9f0377219b474143b09431f000d38f28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:35:14.802367 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6fe5f18d-62d9-439a-ad41-c14fdbf0dfe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:14.803012 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-6fe5f18d-62d9-439a-ad41-c14fdbf0dfe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:14.804578 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1063/2125] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:35:14 2022] DELETE /identity/v3/projects/9f0377219b474143b09431f000d38f28 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:14.805962 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5332123-905d-4092-a054-d6898aa42a7e None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:14.807276 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5332123-905d-4092-a054-d6898aa42a7e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:14.807574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5332123-905d-4092-a054-d6898aa42a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:14.825232 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f5332123-905d-4092-a054-d6898aa42a7e None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:14.826417 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f5332123-905d-4092-a054-d6898aa42a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:14.833944 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1064/2126] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:14.834829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5332123-905d-4092-a054-d6898aa42a7e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:14.836042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f5332123-905d-4092-a054-d6898aa42a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:14.848527 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5332123-905d-4092-a054-d6898aa42a7e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:14.853169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b0f8ea-22e8-4f8b-8722-bd5d80abf104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:14.854085 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b0f8ea-22e8-4f8b-8722-bd5d80abf104 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:14.854320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b0f8ea-22e8-4f8b-8722-bd5d80abf104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:14.854593 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b0f8ea-22e8-4f8b-8722-bd5d80abf104 admin admin] PATH_INFO: `/v3/projects/a80a4bd1d3764bb184ad239981d1e1f3/users/4d368bff9f91483b81e6a192f7f90ff5/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:14.869221 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b0f8ea-22e8-4f8b-8722-bd5d80abf104 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a80a4bd1d3764bb184ad239981d1e1f3, user_id=4d368bff9f91483b81e6a192f7f90ff5, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:14.871987 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b0f8ea-22e8-4f8b-8722-bd5d80abf104 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:14.944664 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1065/2127] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:35:14 2022] PUT /identity/v3/projects/a80a4bd1d3764bb184ad239981d1e1f3/users/4d368bff9f91483b81e6a192f7f90ff5/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:14.946496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed4a22-763d-45de-8e26-5a9051356a2f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:14.947543 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed4a22-763d-45de-8e26-5a9051356a2f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:14.948068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed4a22-763d-45de-8e26-5a9051356a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:14.964551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2fe421b8-8c05-4e12-98d5-ac075d13610a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf5b88ea871b4c7687576406fede7c31'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:14.966980 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-17ed4a22-763d-45de-8e26-5a9051356a2f None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:14.968270 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-17ed4a22-763d-45de-8e26-5a9051356a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:15.027164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1066/2128] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:15.027967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed4a22-763d-45de-8e26-5a9051356a2f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:15.029221 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-17ed4a22-763d-45de-8e26-5a9051356a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:15.040949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2fe421b8-8c05-4e12-98d5-ac075d13610a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf5b88ea871b4c7687576406fede7c31'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:15.054343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2fe421b8-8c05-4e12-98d5-ac075d13610a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf5b88ea871b4c7687576406fede7c31'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:15.056836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2fe421b8-8c05-4e12-98d5-ac075d13610a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf5b88ea871b4c7687576406fede7c31'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:15.056836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2fe421b8-8c05-4e12-98d5-ac075d13610a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf5b88ea871b4c7687576406fede7c31'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:15.068444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1063/2129] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:35:14 2022] DELETE /identity/v3/users/bf5b88ea871b4c7687576406fede7c31 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:15.069783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a24fd-4059-4605-a8e0-99b3cd5e94e1 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:15.070079 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a24fd-4059-4605-a8e0-99b3cd5e94e1 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:15.070359 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67a24fd-4059-4605-a8e0-99b3cd5e94e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:15.086221 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed4a22-763d-45de-8e26-5a9051356a2f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:15.088119 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-c67a24fd-4059-4605-a8e0-99b3cd5e94e1 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:15.089339 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-c67a24fd-4059-4605-a8e0-99b3cd5e94e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:15.100443 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:15.122518 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:15.138996 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed74c1b-20b8-45f9-9f01-352727de2646 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:15.139666 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed74c1b-20b8-45f9-9f01-352727de2646 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:15.139954 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed74c1b-20b8-45f9-9f01-352727de2646 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:15.140235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed74c1b-20b8-45f9-9f01-352727de2646 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:15.142135 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9ed74c1b-20b8-45f9-9f01-352727de2646 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:15.151144 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1064/2130] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:15.152509 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe03f9-803e-4048-bb9d-52b469c70d77 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:15.152799 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe03f9-803e-4048-bb9d-52b469c70d77 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:15.153074 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbe03f9-803e-4048-bb9d-52b469c70d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:15.156357 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed74c1b-20b8-45f9-9f01-352727de2646 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:15.157830 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed74c1b-20b8-45f9-9f01-352727de2646 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:15.169087 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1067/2131] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:35:15.170434 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f784926-c26b-4763-9980-e6e0a0fafdce None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:15.170774 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f784926-c26b-4763-9980-e6e0a0fafdce None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:15.171056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f784926-c26b-4763-9980-e6e0a0fafdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:15.175128 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-2bbe03f9-803e-4048-bb9d-52b469c70d77 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:15.176382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-2bbe03f9-803e-4048-bb9d-52b469c70d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:15.185591 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1065/2132] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:15.186422 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe03f9-803e-4048-bb9d-52b469c70d77 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:15.189965 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0f784926-c26b-4763-9980-e6e0a0fafdce None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:15.190426 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-2bbe03f9-803e-4048-bb9d-52b469c70d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:15.191167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0f784926-c26b-4763-9980-e6e0a0fafdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:15.200852 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1068/2133] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:15.207654 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f784926-c26b-4763-9980-e6e0a0fafdce None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:15.207789 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbe03f9-803e-4048-bb9d-52b469c70d77 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:15.207984 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143e6e1-23bc-4c13-93ae-d5fa8a73d6e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:15.208880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143e6e1-23bc-4c13-93ae-d5fa8a73d6e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:15.209011 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143e6e1-23bc-4c13-93ae-d5fa8a73d6e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:15.209284 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143e6e1-23bc-4c13-93ae-d5fa8a73d6e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:15.215412 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f784926-c26b-4763-9980-e6e0a0fafdce None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:15.220522 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d3377-63a5-4c56-9cfb-db2cbc4c5a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:15.221301 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d3377-63a5-4c56-9cfb-db2cbc4c5a7d admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:15.221744 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d3377-63a5-4c56-9cfb-db2cbc4c5a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:15.221859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143e6e1-23bc-4c13-93ae-d5fa8a73d6e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:15.222276 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6d3377-63a5-4c56-9cfb-db2cbc4c5a7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:15.223329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6d3377-63a5-4c56-9cfb-db2cbc4c5a7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:35:15.223643 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6d3377-63a5-4c56-9cfb-db2cbc4c5a7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:15.224071 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143e6e1-23bc-4c13-93ae-d5fa8a73d6e2 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:15.225785 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6d3377-63a5-4c56-9cfb-db2cbc4c5a7d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:15.229104 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:15.237530 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1066/2134] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:35:15.238400 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143e6e1-23bc-4c13-93ae-d5fa8a73d6e2 service neutron] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:15.245237 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1069/2135] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:35:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:35:15.246717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d3377-63a5-4c56-9cfb-db2cbc4c5a7d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:15.247590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143e6e1-23bc-4c13-93ae-d5fa8a73d6e2 service neutron] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:15.251533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1b064-6c9a-473b-a146-96c70dff3a59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:15.252194 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1b064-6c9a-473b-a146-96c70dff3a59 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:15.252522 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1b064-6c9a-473b-a146-96c70dff3a59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:15.252803 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe1b064-6c9a-473b-a146-96c70dff3a59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:15.259986 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6d3377-63a5-4c56-9cfb-db2cbc4c5a7d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:15.264375 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1b064-6c9a-473b-a146-96c70dff3a59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:15.268242 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec4f79c-fa1b-434a-b728-289be1975c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:15.269077 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4f79c-fa1b-434a-b728-289be1975c70 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:15.269402 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4f79c-fa1b-434a-b728-289be1975c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:15.269750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec4f79c-fa1b-434a-b728-289be1975c70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:15.271033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec4f79c-fa1b-434a-b728-289be1975c70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:15.273310 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec4f79c-fa1b-434a-b728-289be1975c70 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:15.283957 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe1b064-6c9a-473b-a146-96c70dff3a59 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:15.294803 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1067/2136] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:35:15.425381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1070/2137] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:35:15 2022] POST /identity/v3/projects => generated 373 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:15.431999 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1b064-6c9a-473b-a146-96c70dff3a59 service neutron] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:15.447046 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe1b064-6c9a-473b-a146-96c70dff3a59 service neutron] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:15.527390 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d484b3f-bd63-49c9-b90c-607d0abaf17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:15.528057 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d484b3f-bd63-49c9-b90c-607d0abaf17e admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:15.528345 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d484b3f-bd63-49c9-b90c-607d0abaf17e admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:15.528630 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d484b3f-bd63-49c9-b90c-607d0abaf17e admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:15.529337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d484b3f-bd63-49c9-b90c-607d0abaf17e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:15.531559 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d484b3f-bd63-49c9-b90c-607d0abaf17e admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:15.625562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:15.679135 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1068/2138] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:35:15 2022] POST /identity/v3/users => generated 287 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:15.686014 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec4f79c-fa1b-434a-b728-289be1975c70 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:15.690915 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec4f79c-fa1b-434a-b728-289be1975c70 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:15.695709 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96bd6b-2418-4087-a82c-52a5dadd5250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:15.696389 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96bd6b-2418-4087-a82c-52a5dadd5250 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:15.696712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96bd6b-2418-4087-a82c-52a5dadd5250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:15.697193 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f96bd6b-2418-4087-a82c-52a5dadd5250 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:15.698059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f96bd6b-2418-4087-a82c-52a5dadd5250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:15.699720 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f96bd6b-2418-4087-a82c-52a5dadd5250 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:15.710025 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1071/2139] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:35:15.714534 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d484b3f-bd63-49c9-b90c-607d0abaf17e admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:15.724734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d484b3f-bd63-49c9-b90c-607d0abaf17e admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:15.730089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a35c3a-53d8-4c9a-89f0-a997aa545f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:15.730754 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a35c3a-53d8-4c9a-89f0-a997aa545f58 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:15.731033 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a35c3a-53d8-4c9a-89f0-a997aa545f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:15.731340 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a35c3a-53d8-4c9a-89f0-a997aa545f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:15.732102 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a35c3a-53d8-4c9a-89f0-a997aa545f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:15.733701 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a35c3a-53d8-4c9a-89f0-a997aa545f58 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:15.745304 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1069/2140] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:35:15.749599 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96bd6b-2418-4087-a82c-52a5dadd5250 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:15.761522 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f96bd6b-2418-4087-a82c-52a5dadd5250 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:15.772340 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82332d6a-8c0f-4506-aebb-7cc25a91beeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:15.772340 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82332d6a-8c0f-4506-aebb-7cc25a91beeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:15.772340 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82332d6a-8c0f-4506-aebb-7cc25a91beeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:15.774913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82332d6a-8c0f-4506-aebb-7cc25a91beeb admin admin] PATH_INFO: `/v3/projects/81540485036f472d9ae481a97dc32965/users/c67926b0c8394ca28ebef50230387127/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:15.776189 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a35c3a-53d8-4c9a-89f0-a997aa545f58 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:15.778633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-39a35c3a-53d8-4c9a-89f0-a997aa545f58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:15.796204 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a35c3a-53d8-4c9a-89f0-a997aa545f58 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:15.801857 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77b0113-b905-402a-85de-728a032d7f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:15.801997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82332d6a-8c0f-4506-aebb-7cc25a91beeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=81540485036f472d9ae481a97dc32965, user_id=c67926b0c8394ca28ebef50230387127, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:15.802731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b0113-b905-402a-85de-728a032d7f01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:15.803023 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b0113-b905-402a-85de-728a032d7f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:15.803358 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77b0113-b905-402a-85de-728a032d7f01 admin admin] PATH_INFO: `/v3/projects/a5f9ffff0c414642a9fed631fbf111d4` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:15.804675 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b0113-b905-402a-85de-728a032d7f01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5f9ffff0c414642a9fed631fbf111d4)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:15.804932 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82332d6a-8c0f-4506-aebb-7cc25a91beeb admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:15.807257 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77b0113-b905-402a-85de-728a032d7f01 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:15.920691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1072/2141] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:35:15 2022] PUT /identity/v3/projects/81540485036f472d9ae481a97dc32965/users/c67926b0c8394ca28ebef50230387127/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:15.920691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82332d6a-8c0f-4506-aebb-7cc25a91beeb admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:15.920691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-82332d6a-8c0f-4506-aebb-7cc25a91beeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:15.968310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d77b0113-b905-402a-85de-728a032d7f01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5f9ffff0c414642a9fed631fbf111d4', 'request_id': 'req-d77b0113-b905-402a-85de-728a032d7f01'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:15.974811 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d77b0113-b905-402a-85de-728a032d7f01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5f9ffff0c414642a9fed631fbf111d4', 'request_id': 'req-d77b0113-b905-402a-85de-728a032d7f01'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:15.981595 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d77b0113-b905-402a-85de-728a032d7f01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5f9ffff0c414642a9fed631fbf111d4', 'request_id': 'req-d77b0113-b905-402a-85de-728a032d7f01'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:15.982411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82332d6a-8c0f-4506-aebb-7cc25a91beeb admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:16.037202 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb69b63-8df0-4dd9-b628-275151eadf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:16.037999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb69b63-8df0-4dd9-b628-275151eadf0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:16.038285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb69b63-8df0-4dd9-b628-275151eadf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:16.038563 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb69b63-8df0-4dd9-b628-275151eadf0c admin admin] PATH_INFO: `/v3/projects/ecc2f6432157479e8115fcea39f90cae` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:16.039783 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb69b63-8df0-4dd9-b628-275151eadf0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecc2f6432157479e8115fcea39f90cae)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:16.042383 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb69b63-8df0-4dd9-b628-275151eadf0c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:16.110865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d77b0113-b905-402a-85de-728a032d7f01 admin admin] The token cache is being invalidate because project a5f9ffff0c414642a9fed631fbf111d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:35:16.111161 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d77b0113-b905-402a-85de-728a032d7f01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:16.111856 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d77b0113-b905-402a-85de-728a032d7f01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:16.114225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1070/2142] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:35:15 2022] DELETE /identity/v3/projects/a5f9ffff0c414642a9fed631fbf111d4 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:35:16.116533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cdf29-7042-4adf-ba6c-337492e65953 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:16.117038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cdf29-7042-4adf-ba6c-337492e65953 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:16.117502 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cdf29-7042-4adf-ba6c-337492e65953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:16.148844 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-e24cdf29-7042-4adf-ba6c-337492e65953 None None] MFA Rules not processed for user `c67926b0c8394ca28ebef50230387127`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:16.150529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-e24cdf29-7042-4adf-ba6c-337492e65953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:16.174558 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:16.195078 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3fb69b63-8df0-4dd9-b628-275151eadf0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecc2f6432157479e8115fcea39f90cae', 'request_id': 'req-3fb69b63-8df0-4dd9-b628-275151eadf0c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:16.222767 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1071/2143] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:16.223809 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24cdf29-7042-4adf-ba6c-337492e65953 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:16.225090 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-e24cdf29-7042-4adf-ba6c-337492e65953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:16.239151 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24cdf29-7042-4adf-ba6c-337492e65953 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:16.287728 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3fb69b63-8df0-4dd9-b628-275151eadf0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecc2f6432157479e8115fcea39f90cae', 'request_id': 'req-3fb69b63-8df0-4dd9-b628-275151eadf0c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:16.295532 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe637b1e-a2a7-4343-935c-51d90bdc97fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:16.298048 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3fb69b63-8df0-4dd9-b628-275151eadf0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecc2f6432157479e8115fcea39f90cae', 'request_id': 'req-3fb69b63-8df0-4dd9-b628-275151eadf0c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:16.298167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe637b1e-a2a7-4343-935c-51d90bdc97fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:16.298167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe637b1e-a2a7-4343-935c-51d90bdc97fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:16.298167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe637b1e-a2a7-4343-935c-51d90bdc97fc admin admin] PATH_INFO: `/v3/users/7fc481224937454f8c7c5c8ea7e73031` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:16.298167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe637b1e-a2a7-4343-935c-51d90bdc97fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fc481224937454f8c7c5c8ea7e73031)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:16.300143 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe637b1e-a2a7-4343-935c-51d90bdc97fc admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:16.309213 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3fb69b63-8df0-4dd9-b628-275151eadf0c admin admin] The token cache is being invalidate because project ecc2f6432157479e8115fcea39f90cae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:35:16.309501 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3fb69b63-8df0-4dd9-b628-275151eadf0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:16.310206 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-3fb69b63-8df0-4dd9-b628-275151eadf0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:16.312092 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1073/2144] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:35:15 2022] DELETE /identity/v3/projects/ecc2f6432157479e8115fcea39f90cae => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:16.312755 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb69b63-8df0-4dd9-b628-275151eadf0c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:16.314063 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3fb69b63-8df0-4dd9-b628-275151eadf0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:16.375665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb69b63-8df0-4dd9-b628-275151eadf0c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:16.379454 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c50dc3-7b6c-4a6d-9882-352b19200f3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:16.380105 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c50dc3-7b6c-4a6d-9882-352b19200f3e service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:16.380387 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c50dc3-7b6c-4a6d-9882-352b19200f3e service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:16.380667 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c50dc3-7b6c-4a6d-9882-352b19200f3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:16.382435 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b2c50dc3-7b6c-4a6d-9882-352b19200f3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:16.461296 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c50dc3-7b6c-4a6d-9882-352b19200f3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:16.463441 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c50dc3-7b6c-4a6d-9882-352b19200f3e service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:16.475585 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1074/2145] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:35:16.477375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c50dc3-7b6c-4a6d-9882-352b19200f3e service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:16.481656 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b2c50dc3-7b6c-4a6d-9882-352b19200f3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:16.482619 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-fe637b1e-a2a7-4343-935c-51d90bdc97fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fc481224937454f8c7c5c8ea7e73031'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:16.504238 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c50dc3-7b6c-4a6d-9882-352b19200f3e service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:16.512404 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd754b-ffc5-44f3-9f60-174f38148231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:16.513390 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd754b-ffc5-44f3-9f60-174f38148231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:16.513902 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd754b-ffc5-44f3-9f60-174f38148231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:16.514354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dd754b-ffc5-44f3-9f60-174f38148231 admin admin] PATH_INFO: `/v3/users/353a8bed680d411886462e32483dcaee` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:16.515797 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd754b-ffc5-44f3-9f60-174f38148231 admin admin] RBAC: Authorizing `identity:delete_user(user_id=353a8bed680d411886462e32483dcaee)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:16.518135 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10dd754b-ffc5-44f3-9f60-174f38148231 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:16.573163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-fe637b1e-a2a7-4343-935c-51d90bdc97fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fc481224937454f8c7c5c8ea7e73031'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:16.586916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-fe637b1e-a2a7-4343-935c-51d90bdc97fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fc481224937454f8c7c5c8ea7e73031'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:16.588398 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-fe637b1e-a2a7-4343-935c-51d90bdc97fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fc481224937454f8c7c5c8ea7e73031'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:16.589064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-fe637b1e-a2a7-4343-935c-51d90bdc97fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fc481224937454f8c7c5c8ea7e73031'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:16.602116 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1072/2146] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:35:16 2022] DELETE /identity/v3/users/7fc481224937454f8c7c5c8ea7e73031 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:16.767339 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:16.782337 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-10dd754b-ffc5-44f3-9f60-174f38148231 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '353a8bed680d411886462e32483dcaee'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:16.783317 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-10dd754b-ffc5-44f3-9f60-174f38148231 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '353a8bed680d411886462e32483dcaee'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:16.793975 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-10dd754b-ffc5-44f3-9f60-174f38148231 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '353a8bed680d411886462e32483dcaee'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:16.794673 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-10dd754b-ffc5-44f3-9f60-174f38148231 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '353a8bed680d411886462e32483dcaee'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:16.865578 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:16.866726 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-10dd754b-ffc5-44f3-9f60-174f38148231 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '353a8bed680d411886462e32483dcaee'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:16.890253 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1075/2147] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:35:16 2022] DELETE /identity/v3/users/353a8bed680d411886462e32483dcaee => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:17.032424 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fccbe5f-1fd4-4909-8faa-54e7e4e6a5b3 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:17.033357 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fccbe5f-1fd4-4909-8faa-54e7e4e6a5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:17.033357 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fccbe5f-1fd4-4909-8faa-54e7e4e6a5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:17.071352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd754b-ffc5-44f3-9f60-174f38148231 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:17.072471 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-2fccbe5f-1fd4-4909-8faa-54e7e4e6a5b3 None None] MFA Rules not processed for user `4d368bff9f91483b81e6a192f7f90ff5`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:17.072806 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-10dd754b-ffc5-44f3-9f60-174f38148231 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:17.075530 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-2fccbe5f-1fd4-4909-8faa-54e7e4e6a5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:17.147885 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10dd754b-ffc5-44f3-9f60-174f38148231 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:17.153525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc13183-6a32-4f40-9117-2aad22d68c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:17.154420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc13183-6a32-4f40-9117-2aad22d68c94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:17.154702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc13183-6a32-4f40-9117-2aad22d68c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:17.155231 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc13183-6a32-4f40-9117-2aad22d68c94 admin admin] PATH_INFO: `/v3/projects/89f020ded21d40a7afe5cefad65714a0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:17.159476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc13183-6a32-4f40-9117-2aad22d68c94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=89f020ded21d40a7afe5cefad65714a0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:17.159587 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1073/2148] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:35:17.159700 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc13183-6a32-4f40-9117-2aad22d68c94 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:17.170021 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fccbe5f-1fd4-4909-8faa-54e7e4e6a5b3 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:17.171695 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-2fccbe5f-1fd4-4909-8faa-54e7e4e6a5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:17.246137 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fccbe5f-1fd4-4909-8faa-54e7e4e6a5b3 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:17.251842 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beefc07-0c27-45f1-9121-a4769fe7492e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:17.252504 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beefc07-0c27-45f1-9121-a4769fe7492e service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:17.252800 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beefc07-0c27-45f1-9121-a4769fe7492e service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:17.253070 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beefc07-0c27-45f1-9121-a4769fe7492e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:17.265107 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beefc07-0c27-45f1-9121-a4769fe7492e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:17.267561 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beefc07-0c27-45f1-9121-a4769fe7492e service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:17.284155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1074/2149] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:35:17.306437 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9cc13183-6a32-4f40-9117-2aad22d68c94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89f020ded21d40a7afe5cefad65714a0', 'request_id': 'req-9cc13183-6a32-4f40-9117-2aad22d68c94'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:17.373233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9cc13183-6a32-4f40-9117-2aad22d68c94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89f020ded21d40a7afe5cefad65714a0', 'request_id': 'req-9cc13183-6a32-4f40-9117-2aad22d68c94'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:17.381874 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9cc13183-6a32-4f40-9117-2aad22d68c94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89f020ded21d40a7afe5cefad65714a0', 'request_id': 'req-9cc13183-6a32-4f40-9117-2aad22d68c94'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:17.395633 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9cc13183-6a32-4f40-9117-2aad22d68c94 admin admin] The token cache is being invalidate because project 89f020ded21d40a7afe5cefad65714a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:35:17.396106 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9cc13183-6a32-4f40-9117-2aad22d68c94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:17.396704 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-9cc13183-6a32-4f40-9117-2aad22d68c94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:17.398482 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1076/2150] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:35:17 2022] DELETE /identity/v3/projects/89f020ded21d40a7afe5cefad65714a0 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:17.403919 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b3fc2-df1b-4807-82b1-a384fc170b9d None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:17.404218 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b3fc2-df1b-4807-82b1-a384fc170b9d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:17.404502 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32b3fc2-df1b-4807-82b1-a384fc170b9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:17.425987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-f32b3fc2-df1b-4807-82b1-a384fc170b9d None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:17.427364 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-f32b3fc2-df1b-4807-82b1-a384fc170b9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:17.478424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc13183-6a32-4f40-9117-2aad22d68c94 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:17.479960 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9cc13183-6a32-4f40-9117-2aad22d68c94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:17.501270 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1075/2151] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:17.511494 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3a31c-3df0-43b0-bc52-655afa7cf5c9 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:17.511974 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3a31c-3df0-43b0-bc52-655afa7cf5c9 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:17.512225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3a31c-3df0-43b0-bc52-655afa7cf5c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:17.535366 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-a5e3a31c-3df0-43b0-bc52-655afa7cf5c9 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:17.541136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-a5e3a31c-3df0-43b0-bc52-655afa7cf5c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:17.546657 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1076/2152] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:17.550880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3a31c-3df0-43b0-bc52-655afa7cf5c9 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:17.560838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3a31c-3df0-43b0-bc52-655afa7cf5c9 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:17.566623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6eaad-ece3-4aa9-8847-bffb7bf9ffbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:17.567408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6eaad-ece3-4aa9-8847-bffb7bf9ffbd admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:17.567705 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6eaad-ece3-4aa9-8847-bffb7bf9ffbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:17.567982 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6eaad-ece3-4aa9-8847-bffb7bf9ffbd admin admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:17.568714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe6eaad-ece3-4aa9-8847-bffb7bf9ffbd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:35:17.569024 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe6eaad-ece3-4aa9-8847-bffb7bf9ffbd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:17.570622 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe6eaad-ece3-4aa9-8847-bffb7bf9ffbd admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:17.584279 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1077/2153] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:35:17.584573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc13183-6a32-4f40-9117-2aad22d68c94 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:17.589049 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6eaad-ece3-4aa9-8847-bffb7bf9ffbd admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:17.590564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba24dca-116e-4e44-8a69-c0d344d12bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:17.591513 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba24dca-116e-4e44-8a69-c0d344d12bc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:17.592018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba24dca-116e-4e44-8a69-c0d344d12bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:17.592295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba24dca-116e-4e44-8a69-c0d344d12bc3 admin admin] PATH_INFO: `/v3/projects/6baf268f62fe4df0861aea40f9ec8be4` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:17.593567 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba24dca-116e-4e44-8a69-c0d344d12bc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6baf268f62fe4df0861aea40f9ec8be4)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:17.595592 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba24dca-116e-4e44-8a69-c0d344d12bc3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:17.599190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6eaad-ece3-4aa9-8847-bffb7bf9ffbd admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:17.605086 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f37fe1-1376-4045-9146-9bd68efe8c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:17.607925 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f37fe1-1376-4045-9146-9bd68efe8c3a admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:17.607925 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f37fe1-1376-4045-9146-9bd68efe8c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:17.607925 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f37fe1-1376-4045-9146-9bd68efe8c3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:17.607925 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f37fe1-1376-4045-9146-9bd68efe8c3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:17.608973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f37fe1-1376-4045-9146-9bd68efe8c3a admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:17.710657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-eba24dca-116e-4e44-8a69-c0d344d12bc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6baf268f62fe4df0861aea40f9ec8be4', 'request_id': 'req-eba24dca-116e-4e44-8a69-c0d344d12bc3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:17.732880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1078/2154] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:35:17 2022] POST /identity/v3/projects => generated 381 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:35:17.733895 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f37fe1-1376-4045-9146-9bd68efe8c3a admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:17.797218 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f37fe1-1376-4045-9146-9bd68efe8c3a admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:17.802029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41ac5f-b0ea-43ed-b8bc-ec617d982d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:17.802712 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41ac5f-b0ea-43ed-b8bc-ec617d982d68 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:17.802997 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41ac5f-b0ea-43ed-b8bc-ec617d982d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:17.803292 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41ac5f-b0ea-43ed-b8bc-ec617d982d68 admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:17.803996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41ac5f-b0ea-43ed-b8bc-ec617d982d68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:17.806590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41ac5f-b0ea-43ed-b8bc-ec617d982d68 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:17.830152 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-eba24dca-116e-4e44-8a69-c0d344d12bc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6baf268f62fe4df0861aea40f9ec8be4', 'request_id': 'req-eba24dca-116e-4e44-8a69-c0d344d12bc3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:17.837027 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-eba24dca-116e-4e44-8a69-c0d344d12bc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6baf268f62fe4df0861aea40f9ec8be4', 'request_id': 'req-eba24dca-116e-4e44-8a69-c0d344d12bc3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:17.850587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-eba24dca-116e-4e44-8a69-c0d344d12bc3 admin admin] The token cache is being invalidate because project 6baf268f62fe4df0861aea40f9ec8be4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:35:17.850959 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-eba24dca-116e-4e44-8a69-c0d344d12bc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:17.851823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-eba24dca-116e-4e44-8a69-c0d344d12bc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:17.857873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1077/2155] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:35:17 2022] DELETE /identity/v3/projects/6baf268f62fe4df0861aea40f9ec8be4 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:35:17.859381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc522bc-4f7a-4a5c-bc3d-eef4ccf4a5ca None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:17.863252 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc522bc-4f7a-4a5c-bc3d-eef4ccf4a5ca None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:17.863252 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc522bc-4f7a-4a5c-bc3d-eef4ccf4a5ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:17.879851 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-2cc522bc-4f7a-4a5c-bc3d-eef4ccf4a5ca None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:17.881099 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2cc522bc-4f7a-4a5c-bc3d-eef4ccf4a5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:17.889889 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1078/2156] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:17.899336 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7e3f6-ffad-48d5-9c2d-272c96834912 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:17.899630 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7e3f6-ffad-48d5-9c2d-272c96834912 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:17.900456 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b7e3f6-ffad-48d5-9c2d-272c96834912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:17.918643 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-82b7e3f6-ffad-48d5-9c2d-272c96834912 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:17.919858 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-82b7e3f6-ffad-48d5-9c2d-272c96834912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:17.927792 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1079/2157] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:35:17.932859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7e3f6-ffad-48d5-9c2d-272c96834912 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:17.934943 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1079/2158] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:35:17 2022] POST /identity/v3/users => generated 291 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:35:17.938532 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41ac5f-b0ea-43ed-b8bc-ec617d982d68 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:17.940226 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-cb41ac5f-b0ea-43ed-b8bc-ec617d982d68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:17.942169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b7e3f6-ffad-48d5-9c2d-272c96834912 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:17.948395 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df145ce6-7616-4619-8f51-4b7169637d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:17.949031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df145ce6-7616-4619-8f51-4b7169637d9a admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:17.949317 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df145ce6-7616-4619-8f51-4b7169637d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:17.949594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df145ce6-7616-4619-8f51-4b7169637d9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:17.950404 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df145ce6-7616-4619-8f51-4b7169637d9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:35:17.950770 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df145ce6-7616-4619-8f51-4b7169637d9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:17.952393 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df145ce6-7616-4619-8f51-4b7169637d9a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:17.983439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41ac5f-b0ea-43ed-b8bc-ec617d982d68 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:17.983439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005648c4-0ea3-451d-86d3-5da867e6ba38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:17.983439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005648c4-0ea3-451d-86d3-5da867e6ba38 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:17.983439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005648c4-0ea3-451d-86d3-5da867e6ba38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:17.983846 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005648c4-0ea3-451d-86d3-5da867e6ba38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:17.983846 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005648c4-0ea3-451d-86d3-5da867e6ba38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:17.983846 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005648c4-0ea3-451d-86d3-5da867e6ba38 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:17.984088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1080/2159] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:35:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:35:17.984177 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1080/2160] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:35:17.986335 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df145ce6-7616-4619-8f51-4b7169637d9a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:17.987124 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005648c4-0ea3-451d-86d3-5da867e6ba38 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:17.996457 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df145ce6-7616-4619-8f51-4b7169637d9a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:17.996602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005648c4-0ea3-451d-86d3-5da867e6ba38 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:18.001211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b3349-5c30-4ef4-b2e9-9a77f9051422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:18.001340 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c099c85-8216-4c55-a0ad-a5ed77d024ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:18.001856 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b3349-5c30-4ef4-b2e9-9a77f9051422 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:18.001952 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c099c85-8216-4c55-a0ad-a5ed77d024ca admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:18.002140 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c099c85-8216-4c55-a0ad-a5ed77d024ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:18.002228 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b3349-5c30-4ef4-b2e9-9a77f9051422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:18.002418 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0b3349-5c30-4ef4-b2e9-9a77f9051422 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:18.002516 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c099c85-8216-4c55-a0ad-a5ed77d024ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:18.003138 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b3349-5c30-4ef4-b2e9-9a77f9051422 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:18.003255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c099c85-8216-4c55-a0ad-a5ed77d024ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:18.004776 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c099c85-8216-4c55-a0ad-a5ed77d024ca admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:18.005096 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0b3349-5c30-4ef4-b2e9-9a77f9051422 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:18.018412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1081/2161] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:35:18.033171 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c099c85-8216-4c55-a0ad-a5ed77d024ca admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:18.045062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c099c85-8216-4c55-a0ad-a5ed77d024ca admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:18.049933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb401fbf-4245-428b-a5cd-bf7c87308ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:18.050831 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb401fbf-4245-428b-a5cd-bf7c87308ae5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:18.051104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb401fbf-4245-428b-a5cd-bf7c87308ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:18.051461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb401fbf-4245-428b-a5cd-bf7c87308ae5 admin admin] PATH_INFO: `/v3/projects/2414a4c4fd514aa0955c529f7621fd07/users/cfe62cb2e8264f2db412af52060e1cdf/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:18.070766 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb401fbf-4245-428b-a5cd-bf7c87308ae5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2414a4c4fd514aa0955c529f7621fd07, user_id=cfe62cb2e8264f2db412af52060e1cdf, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:18.076963 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb401fbf-4245-428b-a5cd-bf7c87308ae5 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:18.112516 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1081/2162] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:35:17 2022] POST /identity/v3/projects => generated 367 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:18.114349 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b3349-5c30-4ef4-b2e9-9a77f9051422 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:18.177488 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0b3349-5c30-4ef4-b2e9-9a77f9051422 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:18.183105 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c12c2-81ea-4946-b9af-3883a7615b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:18.183801 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c12c2-81ea-4946-b9af-3883a7615b7d admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:18.184084 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c12c2-81ea-4946-b9af-3883a7615b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:18.184359 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677c12c2-81ea-4946-b9af-3883a7615b7d admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:18.185083 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677c12c2-81ea-4946-b9af-3883a7615b7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:18.187019 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677c12c2-81ea-4946-b9af-3883a7615b7d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:18.208046 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1082/2163] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:35:18 2022] PUT /identity/v3/projects/2414a4c4fd514aa0955c529f7621fd07/users/cfe62cb2e8264f2db412af52060e1cdf/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:18.219502 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08c2ea-b948-48e6-b01f-5c6151be50a7 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:18.219784 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08c2ea-b948-48e6-b01f-5c6151be50a7 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:18.220089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec08c2ea-b948-48e6-b01f-5c6151be50a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:18.251343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-ec08c2ea-b948-48e6-b01f-5c6151be50a7 None None] MFA Rules not processed for user `cfe62cb2e8264f2db412af52060e1cdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:18.251343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-ec08c2ea-b948-48e6-b01f-5c6151be50a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:18.318750 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1083/2164] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:35:18.323953 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08c2ea-b948-48e6-b01f-5c6151be50a7 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:18.340392 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1082/2165] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:35:18 2022] POST /identity/v3/users => generated 284 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:35:18.345297 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c12c2-81ea-4946-b9af-3883a7615b7d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:18.373724 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677c12c2-81ea-4946-b9af-3883a7615b7d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:18.400337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec08c2ea-b948-48e6-b01f-5c6151be50a7 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:18.406188 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c715a1-52bb-454e-a2c0-f97061d61688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:18.406863 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c715a1-52bb-454e-a2c0-f97061d61688 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:18.407148 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c715a1-52bb-454e-a2c0-f97061d61688 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:18.407458 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c715a1-52bb-454e-a2c0-f97061d61688 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:18.408157 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c715a1-52bb-454e-a2c0-f97061d61688 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:18.410083 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c715a1-52bb-454e-a2c0-f97061d61688 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:18.440669 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2915a641-ce05-4bcf-a472-85ee03c03ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:18.441348 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915a641-ce05-4bcf-a472-85ee03c03ffc admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:18.441631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915a641-ce05-4bcf-a472-85ee03c03ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:18.441906 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2915a641-ce05-4bcf-a472-85ee03c03ffc admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:18.442646 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2915a641-ce05-4bcf-a472-85ee03c03ffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:18.444292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2915a641-ce05-4bcf-a472-85ee03c03ffc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:18.454343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1083/2166] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:35:18.455643 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee653b40-1654-4045-bace-e306a1395bd6 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:18.455936 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee653b40-1654-4045-bace-e306a1395bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:18.456216 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee653b40-1654-4045-bace-e306a1395bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:18.473674 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ee653b40-1654-4045-bace-e306a1395bd6 None None] MFA Rules not processed for user `c67926b0c8394ca28ebef50230387127`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:18.474887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ee653b40-1654-4045-bace-e306a1395bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:18.544163 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1084/2167] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:18.544961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee653b40-1654-4045-bace-e306a1395bd6 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:18.557794 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1084/2168] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:35:18 2022] POST /identity/v3/projects => generated 381 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:18.561853 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c715a1-52bb-454e-a2c0-f97061d61688 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:18.570946 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee653b40-1654-4045-bace-e306a1395bd6 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:18.652345 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c715a1-52bb-454e-a2c0-f97061d61688 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:18.653956 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0072210c-7bc6-4fc5-8b2b-ce0f47c20aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:18.654641 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072210c-7bc6-4fc5-8b2b-ce0f47c20aea admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:18.655339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072210c-7bc6-4fc5-8b2b-ce0f47c20aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:18.655630 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0072210c-7bc6-4fc5-8b2b-ce0f47c20aea admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:18.656370 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0072210c-7bc6-4fc5-8b2b-ce0f47c20aea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:18.658024 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0072210c-7bc6-4fc5-8b2b-ce0f47c20aea admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:18.660106 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb75d93-72b9-435e-8baa-e528d6d47a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:18.660996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb75d93-72b9-435e-8baa-e528d6d47a86 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:18.661477 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb75d93-72b9-435e-8baa-e528d6d47a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:18.661879 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb75d93-72b9-435e-8baa-e528d6d47a86 admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:18.662951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb75d93-72b9-435e-8baa-e528d6d47a86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:18.666736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb75d93-72b9-435e-8baa-e528d6d47a86 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:18.669301 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1085/2169] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:35:18.676857 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0072210c-7bc6-4fc5-8b2b-ce0f47c20aea admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:18.691066 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0072210c-7bc6-4fc5-8b2b-ce0f47c20aea admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:18.698786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af8ef58-e515-468a-9876-776cf88ff426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:18.699655 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:18.700262 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8ef58-e515-468a-9876-776cf88ff426 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:18.700556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8ef58-e515-468a-9876-776cf88ff426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:18.700840 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af8ef58-e515-468a-9876-776cf88ff426 admin admin] PATH_INFO: `/v3/projects/c12dcba2816e4a29a50f46072e1f6ad1/users/2e9038415bf34134856900ae1a22a412/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:18.717012 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af8ef58-e515-468a-9876-776cf88ff426 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c12dcba2816e4a29a50f46072e1f6ad1, user_id=2e9038415bf34134856900ae1a22a412, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:18.719743 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af8ef58-e515-468a-9876-776cf88ff426 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:18.756429 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1085/2170] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:35:18 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:18.760441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb75d93-72b9-435e-8baa-e528d6d47a86 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:18.770070 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb75d93-72b9-435e-8baa-e528d6d47a86 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:18.775486 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f126de-19a5-41c8-8d76-ef5dde98758e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:18.776162 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f126de-19a5-41c8-8d76-ef5dde98758e admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:18.776485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f126de-19a5-41c8-8d76-ef5dde98758e admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:18.776920 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f126de-19a5-41c8-8d76-ef5dde98758e admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:18.777666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f126de-19a5-41c8-8d76-ef5dde98758e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:18.779710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f126de-19a5-41c8-8d76-ef5dde98758e admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:18.793260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1086/2171] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:35:18 2022] PUT /identity/v3/projects/c12dcba2816e4a29a50f46072e1f6ad1/users/2e9038415bf34134856900ae1a22a412/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:18.803715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85385d11-9aae-4ebb-b7d9-1475454670e9 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:18.806919 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85385d11-9aae-4ebb-b7d9-1475454670e9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:18.807040 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1086/2172] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:18 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:35:18.807507 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85385d11-9aae-4ebb-b7d9-1475454670e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:18.808192 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f126de-19a5-41c8-8d76-ef5dde98758e admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:18.819637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f126de-19a5-41c8-8d76-ef5dde98758e admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:18.834575 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-85385d11-9aae-4ebb-b7d9-1475454670e9 None None] MFA Rules not processed for user `2e9038415bf34134856900ae1a22a412`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:18.835999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-85385d11-9aae-4ebb-b7d9-1475454670e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:18.899249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d020ef-c308-4f09-8f3b-b4ac0688ddd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:18.899249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d020ef-c308-4f09-8f3b-b4ac0688ddd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:18.899558 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d020ef-c308-4f09-8f3b-b4ac0688ddd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:18.900037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d020ef-c308-4f09-8f3b-b4ac0688ddd5 admin admin] PATH_INFO: `/v3/projects/7aa1881673ac468689649eb3bb6f63af/users/1c804cafce024c1f94ead380f4b23f02/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:18.913649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1087/2173] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:18 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:35:18.930655 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d020ef-c308-4f09-8f3b-b4ac0688ddd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aa1881673ac468689649eb3bb6f63af, user_id=1c804cafce024c1f94ead380f4b23f02, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:18.930762 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85385d11-9aae-4ebb-b7d9-1475454670e9 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:18.930852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d020ef-c308-4f09-8f3b-b4ac0688ddd5 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:18.950712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:18.952442 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85385d11-9aae-4ebb-b7d9-1475454670e9 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:18.968513 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4c877-8c54-4534-ae56-cbb168ee9db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:18.970014 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4c877-8c54-4534-ae56-cbb168ee9db9 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:18.970374 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4c877-8c54-4534-ae56-cbb168ee9db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:18.970764 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4c877-8c54-4534-ae56-cbb168ee9db9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:18.971670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4c877-8c54-4534-ae56-cbb168ee9db9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:18.973854 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4c877-8c54-4534-ae56-cbb168ee9db9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:19.023914 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1087/2174] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:35:18 2022] PUT /identity/v3/projects/7aa1881673ac468689649eb3bb6f63af/users/1c804cafce024c1f94ead380f4b23f02/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:19.030873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d020ef-c308-4f09-8f3b-b4ac0688ddd5 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:19.093247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d020ef-c308-4f09-8f3b-b4ac0688ddd5 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:19.098040 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be365bfa-b345-46c3-a572-cf969750c85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:19.098710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be365bfa-b345-46c3-a572-cf969750c85a admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:19.099016 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be365bfa-b345-46c3-a572-cf969750c85a admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:19.099284 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be365bfa-b345-46c3-a572-cf969750c85a admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:19.100034 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be365bfa-b345-46c3-a572-cf969750c85a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:19.101638 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be365bfa-b345-46c3-a572-cf969750c85a admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:19.117354 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1088/2175] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:35:19.123080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be365bfa-b345-46c3-a572-cf969750c85a admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:19.124168 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1088/2176] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:35:18 2022] POST /identity/v3/projects => generated 367 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:19.126355 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4c877-8c54-4534-ae56-cbb168ee9db9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:19.136639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be365bfa-b345-46c3-a572-cf969750c85a admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:19.234337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce320b7-836b-49c6-b065-824faa31c028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:19.235680 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce320b7-836b-49c6-b065-824faa31c028 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:19.236223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce320b7-836b-49c6-b065-824faa31c028 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:19.236510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce320b7-836b-49c6-b065-824faa31c028 admin admin] PATH_INFO: `/v3/projects/7aa1881673ac468689649eb3bb6f63af/users/1c804cafce024c1f94ead380f4b23f02/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:19.238987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce320b7-836b-49c6-b065-824faa31c028 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aa1881673ac468689649eb3bb6f63af, user_id=1c804cafce024c1f94ead380f4b23f02, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:19.243045 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce320b7-836b-49c6-b065-824faa31c028 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:19.260122 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4c877-8c54-4534-ae56-cbb168ee9db9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:19.271338 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1089/2177] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:35:19 2022] PUT /identity/v3/projects/7aa1881673ac468689649eb3bb6f63af/users/1c804cafce024c1f94ead380f4b23f02/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:35:19.273009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1633a-18a9-497a-a796-b5232a35e704 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:19.273434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1633a-18a9-497a-a796-b5232a35e704 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:19.273965 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1633a-18a9-497a-a796-b5232a35e704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:19.310971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-4fc1633a-18a9-497a-a796-b5232a35e704 None None] MFA Rules not processed for user `4d368bff9f91483b81e6a192f7f90ff5`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:19.312516 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-4fc1633a-18a9-497a-a796-b5232a35e704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:19.369229 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48209f-ddcb-47f6-89f3-775765329380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:19.369901 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48209f-ddcb-47f6-89f3-775765329380 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:19.370187 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48209f-ddcb-47f6-89f3-775765329380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:19.370459 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48209f-ddcb-47f6-89f3-775765329380 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:19.371151 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a48209f-ddcb-47f6-89f3-775765329380 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:19.373155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a48209f-ddcb-47f6-89f3-775765329380 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:19.382471 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1090/2178] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:19.383745 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9ec57-f20c-4120-a842-60a303cb6e7d None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:19.384066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9ec57-f20c-4120-a842-60a303cb6e7d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:19.384459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9ec57-f20c-4120-a842-60a303cb6e7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:19.414392 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-79b9ec57-f20c-4120-a842-60a303cb6e7d None None] MFA Rules not processed for user `1c804cafce024c1f94ead380f4b23f02`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:19.416316 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-79b9ec57-f20c-4120-a842-60a303cb6e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:19.488729 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1091/2179] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:19.504422 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9ec57-f20c-4120-a842-60a303cb6e7d None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:19.506124 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-79b9ec57-f20c-4120-a842-60a303cb6e7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:19.507080 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1089/2180] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:35:19 2022] POST /identity/v3/users => generated 290 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:19.511108 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48209f-ddcb-47f6-89f3-775765329380 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:19.521155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48209f-ddcb-47f6-89f3-775765329380 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:19.527507 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3911adf-6f54-4886-bfaa-5f5475bbdcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:19.528182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3911adf-6f54-4886-bfaa-5f5475bbdcd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:19.528470 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3911adf-6f54-4886-bfaa-5f5475bbdcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:19.528748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3911adf-6f54-4886-bfaa-5f5475bbdcd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:19.529482 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3911adf-6f54-4886-bfaa-5f5475bbdcd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:19.531468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3911adf-6f54-4886-bfaa-5f5475bbdcd6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:19.542885 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1090/2181] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:35:19.547844 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3911adf-6f54-4886-bfaa-5f5475bbdcd6 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:19.558443 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3911adf-6f54-4886-bfaa-5f5475bbdcd6 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:19.563772 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e9bae-0300-488c-a9a8-3432b0c1a953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:19.566268 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e9bae-0300-488c-a9a8-3432b0c1a953 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:19.566268 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e9bae-0300-488c-a9a8-3432b0c1a953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:19.566268 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e9bae-0300-488c-a9a8-3432b0c1a953 admin admin] PATH_INFO: `/v3/projects/9b329db1761c4ba1812f17e7b6a80ed3/users/7328ab8cbca640e2b86f6a6ea9ee654f/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:19.570846 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9ec57-f20c-4120-a842-60a303cb6e7d None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:19.584533 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe615c9-eff7-4649-93e8-8964d3946e0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:19.585359 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe615c9-eff7-4649-93e8-8964d3946e0a service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:19.585500 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e9bae-0300-488c-a9a8-3432b0c1a953 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b329db1761c4ba1812f17e7b6a80ed3, user_id=7328ab8cbca640e2b86f6a6ea9ee654f, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:19.585645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe615c9-eff7-4649-93e8-8964d3946e0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:19.586017 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe615c9-eff7-4649-93e8-8964d3946e0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:19.590987 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e9bae-0300-488c-a9a8-3432b0c1a953 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:19.597212 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe615c9-eff7-4649-93e8-8964d3946e0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:19.600656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe615c9-eff7-4649-93e8-8964d3946e0a service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:19.609109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1092/2182] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:35:19.688685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1091/2183] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:35:19 2022] PUT /identity/v3/projects/9b329db1761c4ba1812f17e7b6a80ed3/users/7328ab8cbca640e2b86f6a6ea9ee654f/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:35:19.692798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe615c9-eff7-4649-93e8-8964d3946e0a service cinder] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:19.704419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe615c9-eff7-4649-93e8-8964d3946e0a service cinder] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:19.769414 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277218ca-d611-4c35-9c18-b1c90d8ab894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:19.770096 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277218ca-d611-4c35-9c18-b1c90d8ab894 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:19.770884 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277218ca-d611-4c35-9c18-b1c90d8ab894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:19.771172 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277218ca-d611-4c35-9c18-b1c90d8ab894 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:19.771943 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277218ca-d611-4c35-9c18-b1c90d8ab894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:19.773619 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277218ca-d611-4c35-9c18-b1c90d8ab894 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:19.787418 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1093/2184] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:35:19.788604 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e9bae-0300-488c-a9a8-3432b0c1a953 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:19.799574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e9bae-0300-488c-a9a8-3432b0c1a953 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:19.804169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582da8bf-79e2-4ac4-87df-2c7617ab844f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:19.805334 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582da8bf-79e2-4ac4-87df-2c7617ab844f admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:19.805919 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582da8bf-79e2-4ac4-87df-2c7617ab844f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:19.806574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582da8bf-79e2-4ac4-87df-2c7617ab844f admin admin] PATH_INFO: `/v3/projects/9b329db1761c4ba1812f17e7b6a80ed3/users/7328ab8cbca640e2b86f6a6ea9ee654f/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:19.809967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582da8bf-79e2-4ac4-87df-2c7617ab844f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b329db1761c4ba1812f17e7b6a80ed3, user_id=7328ab8cbca640e2b86f6a6ea9ee654f, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:19.819864 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582da8bf-79e2-4ac4-87df-2c7617ab844f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:19.827664 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1092/2185] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:35:19 2022] PUT /identity/v3/projects/9b329db1761c4ba1812f17e7b6a80ed3/users/7328ab8cbca640e2b86f6a6ea9ee654f/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:19.838365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d51bb4-40c6-481f-ae94-28741bf24526 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:19.838872 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d51bb4-40c6-481f-ae94-28741bf24526 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:19.838975 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d51bb4-40c6-481f-ae94-28741bf24526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:19.898065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-e8d51bb4-40c6-481f-ae94-28741bf24526 None None] MFA Rules not processed for user `7328ab8cbca640e2b86f6a6ea9ee654f`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:19.899380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-e8d51bb4-40c6-481f-ae94-28741bf24526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:19.968375 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1094/2186] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:19 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:19.980186 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582da8bf-79e2-4ac4-87df-2c7617ab844f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:20.038319 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582da8bf-79e2-4ac4-87df-2c7617ab844f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:20.042505 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d1a63-e97d-4f1a-a2db-42f3babb0e8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:20.043148 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d1a63-e97d-4f1a-a2db-42f3babb0e8c service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:20.044674 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d1a63-e97d-4f1a-a2db-42f3babb0e8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:20.044674 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00d1a63-e97d-4f1a-a2db-42f3babb0e8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:20.052770 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00d1a63-e97d-4f1a-a2db-42f3babb0e8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:20.054211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00d1a63-e97d-4f1a-a2db-42f3babb0e8c service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:20.064730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1093/2187] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:19 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:35:20.401833 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d51bb4-40c6-481f-ae94-28741bf24526 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:20.411362 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d51bb4-40c6-481f-ae94-28741bf24526 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:20.417010 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597adc49-ad9d-45fc-883f-4dfab4babb5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:20.417010 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597adc49-ad9d-45fc-883f-4dfab4babb5b service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:20.417010 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597adc49-ad9d-45fc-883f-4dfab4babb5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:20.417010 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597adc49-ad9d-45fc-883f-4dfab4babb5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:20.472676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597adc49-ad9d-45fc-883f-4dfab4babb5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:20.474122 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597adc49-ad9d-45fc-883f-4dfab4babb5b service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:20.483663 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1095/2188] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:20 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:35:20.522407 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:20.835836 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d1a63-e97d-4f1a-a2db-42f3babb0e8c service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:20.847679 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:20.902854 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00d1a63-e97d-4f1a-a2db-42f3babb0e8c service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:20.907637 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444c4e7-e1c1-4d25-aea7-dd234e6991e2 tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c804cafce024c1f94ead380f4b23f02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7aa1881673ac468689649eb3bb6f63af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:20.908298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444c4e7-e1c1-4d25-aea7-dd234e6991e2 tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:20.908593 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444c4e7-e1c1-4d25-aea7-dd234e6991e2 tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:20.908879 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444c4e7-e1c1-4d25-aea7-dd234e6991e2 tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:20.909611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444c4e7-e1c1-4d25-aea7-dd234e6991e2 tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:35:20.909925 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444c4e7-e1c1-4d25-aea7-dd234e6991e2 tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:20.911561 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444c4e7-e1c1-4d25-aea7-dd234e6991e2 tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:20.923739 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1094/2189] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:35:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:35:20.928097 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597adc49-ad9d-45fc-883f-4dfab4babb5b service cinder] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:20.939172 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597adc49-ad9d-45fc-883f-4dfab4babb5b service cinder] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:20.943766 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cae35f-0688-48ed-8a76-4a2e2847ba9e tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c804cafce024c1f94ead380f4b23f02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7aa1881673ac468689649eb3bb6f63af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:20.944415 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cae35f-0688-48ed-8a76-4a2e2847ba9e tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:20.944699 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cae35f-0688-48ed-8a76-4a2e2847ba9e tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:20.944977 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cae35f-0688-48ed-8a76-4a2e2847ba9e tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:20.945668 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cae35f-0688-48ed-8a76-4a2e2847ba9e tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:20.947813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cae35f-0688-48ed-8a76-4a2e2847ba9e tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:21.072277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1096/2190] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:35:20 2022] POST /identity/v3/projects => generated 355 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:21.076729 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444c4e7-e1c1-4d25-aea7-dd234e6991e2 tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:21.086765 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444c4e7-e1c1-4d25-aea7-dd234e6991e2 tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:21.147770 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb1af1-91b6-47b1-af52-8f70f0aeef5b tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c804cafce024c1f94ead380f4b23f02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7aa1881673ac468689649eb3bb6f63af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:21.148506 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb1af1-91b6-47b1-af52-8f70f0aeef5b tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:21.148799 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb1af1-91b6-47b1-af52-8f70f0aeef5b tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:21.149108 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb1af1-91b6-47b1-af52-8f70f0aeef5b tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:21.150047 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb1af1-91b6-47b1-af52-8f70f0aeef5b tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:35:21.150490 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb1af1-91b6-47b1-af52-8f70f0aeef5b tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:21.152702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbb1af1-91b6-47b1-af52-8f70f0aeef5b tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:21.168300 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1095/2191] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:35:21.217275 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cae35f-0688-48ed-8a76-4a2e2847ba9e tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:21.228250 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cae35f-0688-48ed-8a76-4a2e2847ba9e tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:21.233053 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b66911-1596-444d-a2b3-6e1788f74e2d tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1c804cafce024c1f94ead380f4b23f02', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7aa1881673ac468689649eb3bb6f63af', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:21.233920 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b66911-1596-444d-a2b3-6e1788f74e2d tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:21.234215 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b66911-1596-444d-a2b3-6e1788f74e2d tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:21.234928 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b66911-1596-444d-a2b3-6e1788f74e2d tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] PATH_INFO: `/v3/projects/9dd4b332d1224c18b1bd86387ecd1503` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:21.235720 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b66911-1596-444d-a2b3-6e1788f74e2d tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] RBAC: Authorizing `identity:delete_project(project_id=9dd4b332d1224c18b1bd86387ecd1503)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:21.237636 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b66911-1596-444d-a2b3-6e1788f74e2d tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:21.382409 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a6b66911-1596-444d-a2b3-6e1788f74e2d tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dd4b332d1224c18b1bd86387ecd1503', 'request_id': 'req-a6b66911-1596-444d-a2b3-6e1788f74e2d'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:21.392836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a6b66911-1596-444d-a2b3-6e1788f74e2d tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dd4b332d1224c18b1bd86387ecd1503', 'request_id': 'req-a6b66911-1596-444d-a2b3-6e1788f74e2d'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:21.399392 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a6b66911-1596-444d-a2b3-6e1788f74e2d tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dd4b332d1224c18b1bd86387ecd1503', 'request_id': 'req-a6b66911-1596-444d-a2b3-6e1788f74e2d'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:21.499441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a6b66911-1596-444d-a2b3-6e1788f74e2d tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] The token cache is being invalidate because project 9dd4b332d1224c18b1bd86387ecd1503 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:35:21.499737 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a6b66911-1596-444d-a2b3-6e1788f74e2d tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:21.500413 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a6b66911-1596-444d-a2b3-6e1788f74e2d tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:21.502375 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1097/2192] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:35:21 2022] DELETE /identity/v3/projects/9dd4b332d1224c18b1bd86387ecd1503 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:21.589572 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:22.024927 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:22.094905 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:22.152366 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb1af1-91b6-47b1-af52-8f70f0aeef5b tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:22.154184 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3bbb1af1-91b6-47b1-af52-8f70f0aeef5b tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:22.219136 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:22.223610 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb1af1-91b6-47b1-af52-8f70f0aeef5b tempest-VolumeQuotaClassesTest-1539131357 tempest-VolumeQuotaClassesTest-1539131357-project-admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:22.236316 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:22.281244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e27f6-000c-428f-bc53-98663535321b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:22.282120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e27f6-000c-428f-bc53-98663535321b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:22.282405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e27f6-000c-428f-bc53-98663535321b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:22.282687 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e27f6-000c-428f-bc53-98663535321b admin admin] PATH_INFO: `/v3/users/cfe62cb2e8264f2db412af52060e1cdf` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:22.284013 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e27f6-000c-428f-bc53-98663535321b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfe62cb2e8264f2db412af52060e1cdf)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:22.286035 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e27f6-000c-428f-bc53-98663535321b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:22.288940 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:22.304766 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:22.421018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2e6e27f6-000c-428f-bc53-98663535321b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfe62cb2e8264f2db412af52060e1cdf'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:22.421670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2e6e27f6-000c-428f-bc53-98663535321b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfe62cb2e8264f2db412af52060e1cdf'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:22.428880 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2e6e27f6-000c-428f-bc53-98663535321b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfe62cb2e8264f2db412af52060e1cdf'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:22.429533 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2e6e27f6-000c-428f-bc53-98663535321b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfe62cb2e8264f2db412af52060e1cdf'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:22.492617 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2e6e27f6-000c-428f-bc53-98663535321b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfe62cb2e8264f2db412af52060e1cdf'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:22.521646 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1096/2193] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:35:22 2022] DELETE /identity/v3/users/cfe62cb2e8264f2db412af52060e1cdf => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:22.526765 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581745d5-86a2-4fb1-9c84-a9dcfd1ecc68 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:22.527950 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581745d5-86a2-4fb1-9c84-a9dcfd1ecc68 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:22.528244 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581745d5-86a2-4fb1-9c84-a9dcfd1ecc68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:22.546888 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-581745d5-86a2-4fb1-9c84-a9dcfd1ecc68 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:22.548325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-581745d5-86a2-4fb1-9c84-a9dcfd1ecc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:22.614093 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1098/2194] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:35:22.624818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e27f6-000c-428f-bc53-98663535321b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:22.626248 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2e6e27f6-000c-428f-bc53-98663535321b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:22.631502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:22.684726 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e27f6-000c-428f-bc53-98663535321b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:22.688372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932a873-4b08-48f7-85ef-cbe89794152b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:22.689014 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932a873-4b08-48f7-85ef-cbe89794152b service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:22.689301 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932a873-4b08-48f7-85ef-cbe89794152b service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:22.689580 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7932a873-4b08-48f7-85ef-cbe89794152b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:22.699302 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932a873-4b08-48f7-85ef-cbe89794152b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:22.701340 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7932a873-4b08-48f7-85ef-cbe89794152b service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:22.710609 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1097/2195] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:35:23.186231 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581745d5-86a2-4fb1-9c84-a9dcfd1ecc68 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:23.187766 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-581745d5-86a2-4fb1-9c84-a9dcfd1ecc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:23.201666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581745d5-86a2-4fb1-9c84-a9dcfd1ecc68 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:23.206540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40a65d-526c-4a57-bbf0-d2da533be2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:23.207472 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40a65d-526c-4a57-bbf0-d2da533be2cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:23.207756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40a65d-526c-4a57-bbf0-d2da533be2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:23.208028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40a65d-526c-4a57-bbf0-d2da533be2cc admin admin] PATH_INFO: `/v3/projects/2414a4c4fd514aa0955c529f7621fd07` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:23.210816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40a65d-526c-4a57-bbf0-d2da533be2cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=2414a4c4fd514aa0955c529f7621fd07)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:23.211253 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40a65d-526c-4a57-bbf0-d2da533be2cc admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:23.214515 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:23.330610 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-7b40a65d-526c-4a57-bbf0-d2da533be2cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2414a4c4fd514aa0955c529f7621fd07', 'request_id': 'req-7b40a65d-526c-4a57-bbf0-d2da533be2cc'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:23.338493 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-7b40a65d-526c-4a57-bbf0-d2da533be2cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2414a4c4fd514aa0955c529f7621fd07', 'request_id': 'req-7b40a65d-526c-4a57-bbf0-d2da533be2cc'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:23.345029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-7b40a65d-526c-4a57-bbf0-d2da533be2cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2414a4c4fd514aa0955c529f7621fd07', 'request_id': 'req-7b40a65d-526c-4a57-bbf0-d2da533be2cc'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:23.439494 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-7b40a65d-526c-4a57-bbf0-d2da533be2cc admin admin] The token cache is being invalidate because project 2414a4c4fd514aa0955c529f7621fd07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:35:23.439889 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-7b40a65d-526c-4a57-bbf0-d2da533be2cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:23.443388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-7b40a65d-526c-4a57-bbf0-d2da533be2cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:23.445098 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1099/2196] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:35:23 2022] DELETE /identity/v3/projects/2414a4c4fd514aa0955c529f7621fd07 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:23.450372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932a873-4b08-48f7-85ef-cbe89794152b service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:23.464249 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7932a873-4b08-48f7-85ef-cbe89794152b service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:23.561548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f02700-5393-4980-9ade-a93876ce46e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:23.562477 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f02700-5393-4980-9ade-a93876ce46e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:23.562733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f02700-5393-4980-9ade-a93876ce46e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:23.563011 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f02700-5393-4980-9ade-a93876ce46e8 admin admin] PATH_INFO: `/v3/users/1c804cafce024c1f94ead380f4b23f02` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:23.568371 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f02700-5393-4980-9ade-a93876ce46e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c804cafce024c1f94ead380f4b23f02)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:23.570401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f02700-5393-4980-9ade-a93876ce46e8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:23.833756 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-66f02700-5393-4980-9ade-a93876ce46e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c804cafce024c1f94ead380f4b23f02'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:23.834435 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-66f02700-5393-4980-9ade-a93876ce46e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c804cafce024c1f94ead380f4b23f02'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:23.842287 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-66f02700-5393-4980-9ade-a93876ce46e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c804cafce024c1f94ead380f4b23f02'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:23.843015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-66f02700-5393-4980-9ade-a93876ce46e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c804cafce024c1f94ead380f4b23f02'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:23.932793 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-66f02700-5393-4980-9ade-a93876ce46e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c804cafce024c1f94ead380f4b23f02'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:23.979770 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1098/2197] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:35:23 2022] DELETE /identity/v3/users/1c804cafce024c1f94ead380f4b23f02 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:24.811523 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40a65d-526c-4a57-bbf0-d2da533be2cc admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:24.814629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-7b40a65d-526c-4a57-bbf0-d2da533be2cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:24.856647 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40a65d-526c-4a57-bbf0-d2da533be2cc admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:24.957283 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb6171-f6c6-4939-a66a-34aa4023bb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:24.958353 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb6171-f6c6-4939-a66a-34aa4023bb52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:24.958705 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb6171-f6c6-4939-a66a-34aa4023bb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:24.959054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb6171-f6c6-4939-a66a-34aa4023bb52 admin admin] PATH_INFO: `/v3/projects/7aa1881673ac468689649eb3bb6f63af` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:24.960436 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb6171-f6c6-4939-a66a-34aa4023bb52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aa1881673ac468689649eb3bb6f63af)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:24.962868 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb6171-f6c6-4939-a66a-34aa4023bb52 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:25.137771 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bfcb6171-f6c6-4939-a66a-34aa4023bb52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aa1881673ac468689649eb3bb6f63af', 'request_id': 'req-bfcb6171-f6c6-4939-a66a-34aa4023bb52'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:25.145154 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bfcb6171-f6c6-4939-a66a-34aa4023bb52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aa1881673ac468689649eb3bb6f63af', 'request_id': 'req-bfcb6171-f6c6-4939-a66a-34aa4023bb52'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:25.151369 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bfcb6171-f6c6-4939-a66a-34aa4023bb52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aa1881673ac468689649eb3bb6f63af', 'request_id': 'req-bfcb6171-f6c6-4939-a66a-34aa4023bb52'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:25.253336 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bfcb6171-f6c6-4939-a66a-34aa4023bb52 admin admin] The token cache is being invalidate because project 7aa1881673ac468689649eb3bb6f63af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:35:25.253666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bfcb6171-f6c6-4939-a66a-34aa4023bb52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:25.257760 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bfcb6171-f6c6-4939-a66a-34aa4023bb52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:25.260158 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1100/2198] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:35:24 2022] DELETE /identity/v3/projects/7aa1881673ac468689649eb3bb6f63af => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:25.265156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfb16f-0347-41aa-aa4f-cbbfc2cae277 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:25.268693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfb16f-0347-41aa-aa4f-cbbfc2cae277 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:25.268693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebfb16f-0347-41aa-aa4f-cbbfc2cae277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:25.289421 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3ebfb16f-0347-41aa-aa4f-cbbfc2cae277 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:25.291021 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3ebfb16f-0347-41aa-aa4f-cbbfc2cae277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:25.314106 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:25.383429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1099/2199] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:25.388747 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804281c-e4e2-4331-af3c-ca2fd3da4791 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:25.389129 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804281c-e4e2-4331-af3c-ca2fd3da4791 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:25.389423 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804281c-e4e2-4331-af3c-ca2fd3da4791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:25.420697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-6804281c-e4e2-4331-af3c-ca2fd3da4791 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:25.421936 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-6804281c-e4e2-4331-af3c-ca2fd3da4791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:25.432504 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1101/2200] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:25.436850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfb16f-0347-41aa-aa4f-cbbfc2cae277 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:25.453035 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebfb16f-0347-41aa-aa4f-cbbfc2cae277 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:25.462790 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af7b27-5037-41ee-972d-916d297dd4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:25.463996 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af7b27-5037-41ee-972d-916d297dd4c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:25.464395 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af7b27-5037-41ee-972d-916d297dd4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:25.465004 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af7b27-5037-41ee-972d-916d297dd4c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:25.466569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af7b27-5037-41ee-972d-916d297dd4c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:35:25.466963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af7b27-5037-41ee-972d-916d297dd4c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:25.469342 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af7b27-5037-41ee-972d-916d297dd4c8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:25.484848 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1100/2201] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:35:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:35:25.489219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804281c-e4e2-4331-af3c-ca2fd3da4791 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:25.499916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804281c-e4e2-4331-af3c-ca2fd3da4791 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:25.504654 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699cd5c4-86d6-4c95-91b0-1e6907e28c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:25.505324 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cd5c4-86d6-4c95-91b0-1e6907e28c4f admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:25.505608 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cd5c4-86d6-4c95-91b0-1e6907e28c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:25.505882 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699cd5c4-86d6-4c95-91b0-1e6907e28c4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:25.506580 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699cd5c4-86d6-4c95-91b0-1e6907e28c4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:25.508548 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699cd5c4-86d6-4c95-91b0-1e6907e28c4f admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:25.523338 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:25.589701 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1102/2202] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:35:25 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:25.593680 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af7b27-5037-41ee-972d-916d297dd4c8 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:25.608467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af7b27-5037-41ee-972d-916d297dd4c8 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:25.715084 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9519ed8-5df0-4657-9f5f-1b8e3b03f759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:25.716618 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9519ed8-5df0-4657-9f5f-1b8e3b03f759 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:25.716618 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9519ed8-5df0-4657-9f5f-1b8e3b03f759 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:25.716618 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9519ed8-5df0-4657-9f5f-1b8e3b03f759 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:25.717015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9519ed8-5df0-4657-9f5f-1b8e3b03f759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:25.720147 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9519ed8-5df0-4657-9f5f-1b8e3b03f759 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:25.840541 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1101/2203] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:35:25 2022] POST /identity/v3/users => generated 287 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:25.844542 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699cd5c4-86d6-4c95-91b0-1e6907e28c4f admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:25.857900 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699cd5c4-86d6-4c95-91b0-1e6907e28c4f admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:25.863228 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a3142-e51f-43f1-84a7-56d41175bb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:25.864175 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a3142-e51f-43f1-84a7-56d41175bb2e admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:25.864689 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a3142-e51f-43f1-84a7-56d41175bb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:25.864979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198a3142-e51f-43f1-84a7-56d41175bb2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:25.865732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a3142-e51f-43f1-84a7-56d41175bb2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:25.867616 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198a3142-e51f-43f1-84a7-56d41175bb2e admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:25.881138 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1103/2204] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:35:25.886659 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9519ed8-5df0-4657-9f5f-1b8e3b03f759 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:25.899155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9519ed8-5df0-4657-9f5f-1b8e3b03f759 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:25.906186 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3511945-5689-4f39-90cf-04a1dcda5aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:25.907111 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3511945-5689-4f39-90cf-04a1dcda5aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:25.907585 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3511945-5689-4f39-90cf-04a1dcda5aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:25.913516 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3511945-5689-4f39-90cf-04a1dcda5aa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:25.914327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3511945-5689-4f39-90cf-04a1dcda5aa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:25.916766 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3511945-5689-4f39-90cf-04a1dcda5aa9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:25.943829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1102/2205] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:25 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:35:25.945685 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a3142-e51f-43f1-84a7-56d41175bb2e admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:25.959855 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198a3142-e51f-43f1-84a7-56d41175bb2e admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:25.967678 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266e0d3-5dfc-4338-b568-b67851e25715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:25.971918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266e0d3-5dfc-4338-b568-b67851e25715 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:25.971918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266e0d3-5dfc-4338-b568-b67851e25715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:25.971918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7266e0d3-5dfc-4338-b568-b67851e25715 admin admin] PATH_INFO: `/v3/projects/f13bcc19971147f2a4acaaf8defd31f7/users/168faf704f274bcfb2d40aff94c68c5b/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:25.997544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266e0d3-5dfc-4338-b568-b67851e25715 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f13bcc19971147f2a4acaaf8defd31f7, user_id=168faf704f274bcfb2d40aff94c68c5b, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:26.001539 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7266e0d3-5dfc-4338-b568-b67851e25715 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:26.158255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1104/2206] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:35:25 2022] PUT /identity/v3/projects/f13bcc19971147f2a4acaaf8defd31f7/users/168faf704f274bcfb2d40aff94c68c5b/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:26.171029 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792cc32-f024-4588-9053-2eafd90231c1 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:26.171354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792cc32-f024-4588-9053-2eafd90231c1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:26.172022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1792cc32-f024-4588-9053-2eafd90231c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:26.208327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1792cc32-f024-4588-9053-2eafd90231c1 None None] MFA Rules not processed for user `168faf704f274bcfb2d40aff94c68c5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:26.210372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1792cc32-f024-4588-9053-2eafd90231c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:26.326388 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1103/2207] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:26.328532 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266e0d3-5dfc-4338-b568-b67851e25715 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:26.470636 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7266e0d3-5dfc-4338-b568-b67851e25715 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:26.488358 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574c215-dfdf-4393-8fcc-6876b7834824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:26.488358 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574c215-dfdf-4393-8fcc-6876b7834824 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:26.488358 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574c215-dfdf-4393-8fcc-6876b7834824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:26.488358 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a574c215-dfdf-4393-8fcc-6876b7834824 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:26.488710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a574c215-dfdf-4393-8fcc-6876b7834824 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:26.488710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a574c215-dfdf-4393-8fcc-6876b7834824 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:26.620818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792cc32-f024-4588-9053-2eafd90231c1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:26.630245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1105/2208] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:35:26 2022] POST /identity/v3/projects => generated 375 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:35:26.633333 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1792cc32-f024-4588-9053-2eafd90231c1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:26.692332 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:26.769572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa36f1-7e15-4268-ad1c-d6f3304bffb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:26.770540 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa36f1-7e15-4268-ad1c-d6f3304bffb3 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:26.770900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa36f1-7e15-4268-ad1c-d6f3304bffb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:26.771274 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa36f1-7e15-4268-ad1c-d6f3304bffb3 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:26.772789 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa36f1-7e15-4268-ad1c-d6f3304bffb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:26.776706 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa36f1-7e15-4268-ad1c-d6f3304bffb3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:26.867940 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1104/2209] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:35:26 2022] POST /identity/v3/users => generated 294 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:26.872548 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574c215-dfdf-4393-8fcc-6876b7834824 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:26.890270 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a574c215-dfdf-4393-8fcc-6876b7834824 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:26.912438 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a498d55-04d1-4943-bdd5-1eeb0f7ca00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:26.913527 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a498d55-04d1-4943-bdd5-1eeb0f7ca00a admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:26.914059 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a498d55-04d1-4943-bdd5-1eeb0f7ca00a admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:26.918936 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a498d55-04d1-4943-bdd5-1eeb0f7ca00a admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:26.921320 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a498d55-04d1-4943-bdd5-1eeb0f7ca00a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:26.924189 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a498d55-04d1-4943-bdd5-1eeb0f7ca00a admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:26.941944 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1106/2210] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:26 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:35:26.946507 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa36f1-7e15-4268-ad1c-d6f3304bffb3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:26.972913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa36f1-7e15-4268-ad1c-d6f3304bffb3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:27.001817 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d66a5-ded0-4448-9bc6-54694082fbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:27.002746 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d66a5-ded0-4448-9bc6-54694082fbeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:27.003035 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d66a5-ded0-4448-9bc6-54694082fbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:27.003354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4d66a5-ded0-4448-9bc6-54694082fbeb admin admin] PATH_INFO: `/v3/projects/92dc78f8532b441e9a37c74c6a15ef18/users/e35eb67dd08545d1857e8053de8583e9/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:27.052560 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d66a5-ded0-4448-9bc6-54694082fbeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=92dc78f8532b441e9a37c74c6a15ef18, user_id=e35eb67dd08545d1857e8053de8583e9, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:27.055969 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4d66a5-ded0-4448-9bc6-54694082fbeb admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:27.192146 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1105/2211] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:35:26 2022] PUT /identity/v3/projects/92dc78f8532b441e9a37c74c6a15ef18/users/e35eb67dd08545d1857e8053de8583e9/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:27.196307 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a498d55-04d1-4943-bdd5-1eeb0f7ca00a admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:27.208560 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a498d55-04d1-4943-bdd5-1eeb0f7ca00a admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:27.307255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad062970-0ae3-4102-8885-20f18027d4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:27.308819 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad062970-0ae3-4102-8885-20f18027d4a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:27.309577 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad062970-0ae3-4102-8885-20f18027d4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:27.310464 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad062970-0ae3-4102-8885-20f18027d4a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:27.311985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad062970-0ae3-4102-8885-20f18027d4a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:27.313785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad062970-0ae3-4102-8885-20f18027d4a8 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:27.341288 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1107/2212] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:27 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:35:27.341392 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d66a5-ded0-4448-9bc6-54694082fbeb admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:27.359303 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4d66a5-ded0-4448-9bc6-54694082fbeb admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:27.367712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd63c4a-639e-4fc7-8a01-d3f43536da54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:27.372033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd63c4a-639e-4fc7-8a01-d3f43536da54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:27.372033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd63c4a-639e-4fc7-8a01-d3f43536da54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:27.372033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd63c4a-639e-4fc7-8a01-d3f43536da54 admin admin] PATH_INFO: `/v3/projects/92dc78f8532b441e9a37c74c6a15ef18/users/e35eb67dd08545d1857e8053de8583e9/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:27.375151 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd63c4a-639e-4fc7-8a01-d3f43536da54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92dc78f8532b441e9a37c74c6a15ef18, user_id=e35eb67dd08545d1857e8053de8583e9, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:27.379066 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd63c4a-639e-4fc7-8a01-d3f43536da54 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:27.393372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1106/2213] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:35:27 2022] PUT /identity/v3/projects/92dc78f8532b441e9a37c74c6a15ef18/users/e35eb67dd08545d1857e8053de8583e9/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:27.406585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84463bf1-f7b1-44b5-9574-81ee079a5ca8 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:27.406585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84463bf1-f7b1-44b5-9574-81ee079a5ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:27.406585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84463bf1-f7b1-44b5-9574-81ee079a5ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:27.441841 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-84463bf1-f7b1-44b5-9574-81ee079a5ca8 None None] MFA Rules not processed for user `e35eb67dd08545d1857e8053de8583e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:27.446997 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-84463bf1-f7b1-44b5-9574-81ee079a5ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:27.461220 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:27.522541 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1108/2214] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:27 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:27.629786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd63c4a-639e-4fc7-8a01-d3f43536da54 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:27.631922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1dd63c4a-639e-4fc7-8a01-d3f43536da54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:27.726424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd63c4a-639e-4fc7-8a01-d3f43536da54 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:27.730849 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7de51c-e854-40cb-8b1e-09a0d4adeb2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:27.731776 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7de51c-e854-40cb-8b1e-09a0d4adeb2c service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:27.732300 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7de51c-e854-40cb-8b1e-09a0d4adeb2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:27.732698 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7de51c-e854-40cb-8b1e-09a0d4adeb2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:27.817765 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7de51c-e854-40cb-8b1e-09a0d4adeb2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:27.819345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7de51c-e854-40cb-8b1e-09a0d4adeb2c service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:27.830481 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1107/2215] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:35:34.255139 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84463bf1-f7b1-44b5-9574-81ee079a5ca8 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:34.256681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-84463bf1-f7b1-44b5-9574-81ee079a5ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:34.385456 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84463bf1-f7b1-44b5-9574-81ee079a5ca8 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:34.390037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd97e722-f446-4c08-b6c0-6d911a0d95c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:34.391050 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97e722-f446-4c08-b6c0-6d911a0d95c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:34.391437 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97e722-f446-4c08-b6c0-6d911a0d95c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:34.391721 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd97e722-f446-4c08-b6c0-6d911a0d95c3 admin admin] PATH_INFO: `/v3/users/2e9038415bf34134856900ae1a22a412` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:34.392928 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97e722-f446-4c08-b6c0-6d911a0d95c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e9038415bf34134856900ae1a22a412)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:34.395021 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd97e722-f446-4c08-b6c0-6d911a0d95c3 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:34.588557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-cd97e722-f446-4c08-b6c0-6d911a0d95c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e9038415bf34134856900ae1a22a412'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:34.677538 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7de51c-e854-40cb-8b1e-09a0d4adeb2c service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:34.682517 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-cd97e722-f446-4c08-b6c0-6d911a0d95c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e9038415bf34134856900ae1a22a412'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:34.686916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:34.700751 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-cd97e722-f446-4c08-b6c0-6d911a0d95c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e9038415bf34134856900ae1a22a412'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:34.701448 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-cd97e722-f446-4c08-b6c0-6d911a0d95c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e9038415bf34134856900ae1a22a412'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:34.702066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-cd97e722-f446-4c08-b6c0-6d911a0d95c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e9038415bf34134856900ae1a22a412'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:34.708386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7de51c-e854-40cb-8b1e-09a0d4adeb2c service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:34.713015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d2641-60b4-4452-9426-ed2cf101eefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:34.714909 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1109/2216] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:35:34 2022] DELETE /identity/v3/users/2e9038415bf34134856900ae1a22a412 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:34.715060 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d2641-60b4-4452-9426-ed2cf101eefa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:34.715321 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d2641-60b4-4452-9426-ed2cf101eefa admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:34.715671 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d2641-60b4-4452-9426-ed2cf101eefa admin admin] PATH_INFO: `/v3/users/168faf704f274bcfb2d40aff94c68c5b` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:34.716923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d2641-60b4-4452-9426-ed2cf101eefa admin admin] RBAC: Authorizing `identity:delete_user(user_id=168faf704f274bcfb2d40aff94c68c5b)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:34.719003 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d2641-60b4-4452-9426-ed2cf101eefa admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:34.719371 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc128b-b36a-452b-801b-0c40d65bac62 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:34.719836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc128b-b36a-452b-801b-0c40d65bac62 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:34.720117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bc128b-b36a-452b-801b-0c40d65bac62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:34.721618 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:34.746971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-a7bc128b-b36a-452b-801b-0c40d65bac62 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:34.748337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-a7bc128b-b36a-452b-801b-0c40d65bac62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:34.808536 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1110/2217] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:35:34.818632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc128b-b36a-452b-801b-0c40d65bac62 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:34.820119 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-a7bc128b-b36a-452b-801b-0c40d65bac62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:34.941545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bc128b-b36a-452b-801b-0c40d65bac62 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:34.945648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-355d2641-60b4-4452-9426-ed2cf101eefa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '168faf704f274bcfb2d40aff94c68c5b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:34.946462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38a67fa-f17f-4473-a9d9-d9dd10b2b3c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:34.947133 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a67fa-f17f-4473-a9d9-d9dd10b2b3c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:34.947375 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-355d2641-60b4-4452-9426-ed2cf101eefa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '168faf704f274bcfb2d40aff94c68c5b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:34.947490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a67fa-f17f-4473-a9d9-d9dd10b2b3c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:34.947724 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38a67fa-f17f-4473-a9d9-d9dd10b2b3c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:34.950858 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-e38a67fa-f17f-4473-a9d9-d9dd10b2b3c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:34.959066 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-355d2641-60b4-4452-9426-ed2cf101eefa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '168faf704f274bcfb2d40aff94c68c5b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:34.959812 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-355d2641-60b4-4452-9426-ed2cf101eefa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '168faf704f274bcfb2d40aff94c68c5b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:34.969450 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38a67fa-f17f-4473-a9d9-d9dd10b2b3c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:34.974494 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38a67fa-f17f-4473-a9d9-d9dd10b2b3c0 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:34.983117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1111/2218] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:35:35.068544 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-355d2641-60b4-4452-9426-ed2cf101eefa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '168faf704f274bcfb2d40aff94c68c5b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:35.089339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1108/2219] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:35:34 2022] DELETE /identity/v3/users/168faf704f274bcfb2d40aff94c68c5b => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:35.094018 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f47b0-fe34-46f7-b6e1-d7d39d816025 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:35.094310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f47b0-fe34-46f7-b6e1-d7d39d816025 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:35.098494 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577f47b0-fe34-46f7-b6e1-d7d39d816025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:35.113017 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-577f47b0-fe34-46f7-b6e1-d7d39d816025 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:35.114230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-577f47b0-fe34-46f7-b6e1-d7d39d816025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:35.145692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:35.176203 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1112/2220] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:35.186625 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d2641-60b4-4452-9426-ed2cf101eefa admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:35.188103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-355d2641-60b4-4452-9426-ed2cf101eefa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:35.255961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d2641-60b4-4452-9426-ed2cf101eefa admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:35.260147 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442706c8-817c-4ed8-a963-a89dc19620d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:35.261362 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442706c8-817c-4ed8-a963-a89dc19620d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:35.261652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442706c8-817c-4ed8-a963-a89dc19620d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:35.261940 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442706c8-817c-4ed8-a963-a89dc19620d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:35.272024 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442706c8-817c-4ed8-a963-a89dc19620d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:35.273668 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442706c8-817c-4ed8-a963-a89dc19620d8 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:35.285390 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1109/2221] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:35:35.513925 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f47b0-fe34-46f7-b6e1-d7d39d816025 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:35.515481 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-577f47b0-fe34-46f7-b6e1-d7d39d816025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:35.531905 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577f47b0-fe34-46f7-b6e1-d7d39d816025 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:35.538246 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2adce-22bf-4921-9a71-21bb4211c801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:35.539222 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2adce-22bf-4921-9a71-21bb4211c801 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:35.539537 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2adce-22bf-4921-9a71-21bb4211c801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:35.539812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec2adce-22bf-4921-9a71-21bb4211c801 admin admin] PATH_INFO: `/v3/projects/c12dcba2816e4a29a50f46072e1f6ad1` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:35.541086 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec2adce-22bf-4921-9a71-21bb4211c801 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c12dcba2816e4a29a50f46072e1f6ad1)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:35.543786 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec2adce-22bf-4921-9a71-21bb4211c801 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:35.707923 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1ec2adce-22bf-4921-9a71-21bb4211c801 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c12dcba2816e4a29a50f46072e1f6ad1', 'request_id': 'req-1ec2adce-22bf-4921-9a71-21bb4211c801'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:35.719137 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1ec2adce-22bf-4921-9a71-21bb4211c801 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c12dcba2816e4a29a50f46072e1f6ad1', 'request_id': 'req-1ec2adce-22bf-4921-9a71-21bb4211c801'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:35.728289 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1ec2adce-22bf-4921-9a71-21bb4211c801 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c12dcba2816e4a29a50f46072e1f6ad1', 'request_id': 'req-1ec2adce-22bf-4921-9a71-21bb4211c801'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:35.864192 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1ec2adce-22bf-4921-9a71-21bb4211c801 admin admin] The token cache is being invalidate because project c12dcba2816e4a29a50f46072e1f6ad1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:35:35.864477 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1ec2adce-22bf-4921-9a71-21bb4211c801 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:35.865098 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1ec2adce-22bf-4921-9a71-21bb4211c801 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:35.866637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1113/2222] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:35:35 2022] DELETE /identity/v3/projects/c12dcba2816e4a29a50f46072e1f6ad1 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:35.871302 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442706c8-817c-4ed8-a963-a89dc19620d8 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:35.881878 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442706c8-817c-4ed8-a963-a89dc19620d8 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:35.909315 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2adce-22bf-4921-9a71-21bb4211c801 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:35.911389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-1ec2adce-22bf-4921-9a71-21bb4211c801 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:35.947910 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79635927-8dae-456e-9869-7064a1f3b05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:35.949485 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79635927-8dae-456e-9869-7064a1f3b05e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:35.949825 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79635927-8dae-456e-9869-7064a1f3b05e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:35.950110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79635927-8dae-456e-9869-7064a1f3b05e admin admin] PATH_INFO: `/v3/users/7328ab8cbca640e2b86f6a6ea9ee654f` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:35.951616 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79635927-8dae-456e-9869-7064a1f3b05e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7328ab8cbca640e2b86f6a6ea9ee654f)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:35.954116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79635927-8dae-456e-9869-7064a1f3b05e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:36.022632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec2adce-22bf-4921-9a71-21bb4211c801 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:36.030074 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433f4f6-f4f5-4115-b0b1-a662e12f79a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:36.031020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433f4f6-f4f5-4115-b0b1-a662e12f79a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:36.031420 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433f4f6-f4f5-4115-b0b1-a662e12f79a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:36.034669 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433f4f6-f4f5-4115-b0b1-a662e12f79a8 admin admin] PATH_INFO: `/v3/projects/f13bcc19971147f2a4acaaf8defd31f7` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:36.034669 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433f4f6-f4f5-4115-b0b1-a662e12f79a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f13bcc19971147f2a4acaaf8defd31f7)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:36.035684 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433f4f6-f4f5-4115-b0b1-a662e12f79a8 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:36.080689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:36.256711 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e433f4f6-f4f5-4115-b0b1-a662e12f79a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f13bcc19971147f2a4acaaf8defd31f7', 'request_id': 'req-e433f4f6-f4f5-4115-b0b1-a662e12f79a8'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:36.257029 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-79635927-8dae-456e-9869-7064a1f3b05e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7328ab8cbca640e2b86f6a6ea9ee654f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:36.257792 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-79635927-8dae-456e-9869-7064a1f3b05e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7328ab8cbca640e2b86f6a6ea9ee654f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:36.264318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e433f4f6-f4f5-4115-b0b1-a662e12f79a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f13bcc19971147f2a4acaaf8defd31f7', 'request_id': 'req-e433f4f6-f4f5-4115-b0b1-a662e12f79a8'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:36.265397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-79635927-8dae-456e-9869-7064a1f3b05e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7328ab8cbca640e2b86f6a6ea9ee654f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:36.266001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-79635927-8dae-456e-9869-7064a1f3b05e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7328ab8cbca640e2b86f6a6ea9ee654f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:36.268877 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:36.272949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e433f4f6-f4f5-4115-b0b1-a662e12f79a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f13bcc19971147f2a4acaaf8defd31f7', 'request_id': 'req-e433f4f6-f4f5-4115-b0b1-a662e12f79a8'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:36.315720 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-79635927-8dae-456e-9869-7064a1f3b05e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7328ab8cbca640e2b86f6a6ea9ee654f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:36.341686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1110/2223] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:35:35 2022] DELETE /identity/v3/users/7328ab8cbca640e2b86f6a6ea9ee654f => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:36.395168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e433f4f6-f4f5-4115-b0b1-a662e12f79a8 admin admin] The token cache is being invalidate because project f13bcc19971147f2a4acaaf8defd31f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:35:36.395605 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e433f4f6-f4f5-4115-b0b1-a662e12f79a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:36.396407 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e433f4f6-f4f5-4115-b0b1-a662e12f79a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:36.398019 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1114/2224] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:35:35 2022] DELETE /identity/v3/projects/f13bcc19971147f2a4acaaf8defd31f7 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:36.402656 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79635927-8dae-456e-9869-7064a1f3b05e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:36.404197 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-79635927-8dae-456e-9869-7064a1f3b05e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:36.476774 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79635927-8dae-456e-9869-7064a1f3b05e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:36.481492 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5823774-e35b-4320-a70b-44c6a22ea56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:36.482371 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5823774-e35b-4320-a70b-44c6a22ea56a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:36.482656 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5823774-e35b-4320-a70b-44c6a22ea56a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:36.482936 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5823774-e35b-4320-a70b-44c6a22ea56a admin admin] PATH_INFO: `/v3/users/e35eb67dd08545d1857e8053de8583e9` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:36.484224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5823774-e35b-4320-a70b-44c6a22ea56a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e35eb67dd08545d1857e8053de8583e9)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:36.486184 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5823774-e35b-4320-a70b-44c6a22ea56a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:36.703681 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b5823774-e35b-4320-a70b-44c6a22ea56a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e35eb67dd08545d1857e8053de8583e9'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:36.704358 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b5823774-e35b-4320-a70b-44c6a22ea56a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e35eb67dd08545d1857e8053de8583e9'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:36.711592 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b5823774-e35b-4320-a70b-44c6a22ea56a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e35eb67dd08545d1857e8053de8583e9'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:36.712274 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b5823774-e35b-4320-a70b-44c6a22ea56a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e35eb67dd08545d1857e8053de8583e9'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:36.770435 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433f4f6-f4f5-4115-b0b1-a662e12f79a8 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:36.772368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-e433f4f6-f4f5-4115-b0b1-a662e12f79a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:36.808098 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b5823774-e35b-4320-a70b-44c6a22ea56a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e35eb67dd08545d1857e8053de8583e9'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:36.831230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1111/2225] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:35:36 2022] DELETE /identity/v3/users/e35eb67dd08545d1857e8053de8583e9 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:36.867214 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433f4f6-f4f5-4115-b0b1-a662e12f79a8 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:36.945085 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53e71f-61b4-4891-8495-ab25fd23d73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:36.945997 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53e71f-61b4-4891-8495-ab25fd23d73f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:36.946290 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53e71f-61b4-4891-8495-ab25fd23d73f admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:36.946574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53e71f-61b4-4891-8495-ab25fd23d73f admin admin] PATH_INFO: `/v3/projects/9b329db1761c4ba1812f17e7b6a80ed3` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:36.948396 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53e71f-61b4-4891-8495-ab25fd23d73f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b329db1761c4ba1812f17e7b6a80ed3)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:36.950679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53e71f-61b4-4891-8495-ab25fd23d73f admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:37.097188 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1b53e71f-61b4-4891-8495-ab25fd23d73f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b329db1761c4ba1812f17e7b6a80ed3', 'request_id': 'req-1b53e71f-61b4-4891-8495-ab25fd23d73f'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:37.104828 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1b53e71f-61b4-4891-8495-ab25fd23d73f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b329db1761c4ba1812f17e7b6a80ed3', 'request_id': 'req-1b53e71f-61b4-4891-8495-ab25fd23d73f'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:37.110923 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1b53e71f-61b4-4891-8495-ab25fd23d73f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b329db1761c4ba1812f17e7b6a80ed3', 'request_id': 'req-1b53e71f-61b4-4891-8495-ab25fd23d73f'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:37.230552 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1b53e71f-61b4-4891-8495-ab25fd23d73f admin admin] The token cache is being invalidate because project 9b329db1761c4ba1812f17e7b6a80ed3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:35:37.230856 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1b53e71f-61b4-4891-8495-ab25fd23d73f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:37.231584 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-1b53e71f-61b4-4891-8495-ab25fd23d73f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:37.233392 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1115/2226] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:35:36 2022] DELETE /identity/v3/projects/9b329db1761c4ba1812f17e7b6a80ed3 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:35:37.238304 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63157f54-cd6f-4220-9e9a-7533231133a5 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:37.238803 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63157f54-cd6f-4220-9e9a-7533231133a5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:37.238973 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63157f54-cd6f-4220-9e9a-7533231133a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:37.300464 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:37.311067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-63157f54-cd6f-4220-9e9a-7533231133a5 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:37.314015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-63157f54-cd6f-4220-9e9a-7533231133a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:37.321506 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53e71f-61b4-4891-8495-ab25fd23d73f admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:37.323668 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-1b53e71f-61b4-4891-8495-ab25fd23d73f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:37.398152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:37.434568 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1112/2227] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:35:37.444994 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f85846-6793-4421-93ed-99d9897a6906 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:37.446083 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f85846-6793-4421-93ed-99d9897a6906 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:37.446083 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f85846-6793-4421-93ed-99d9897a6906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:37.474008 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53e71f-61b4-4891-8495-ab25fd23d73f admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:37.476543 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-25f85846-6793-4421-93ed-99d9897a6906 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:37.478651 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-25f85846-6793-4421-93ed-99d9897a6906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:37.481150 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd61d06f-0053-4e32-9dcd-6cb0e6231350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:37.482499 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61d06f-0053-4e32-9dcd-6cb0e6231350 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:37.483019 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61d06f-0053-4e32-9dcd-6cb0e6231350 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:37.483445 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd61d06f-0053-4e32-9dcd-6cb0e6231350 admin admin] PATH_INFO: `/v3/projects/92dc78f8532b441e9a37c74c6a15ef18` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:37.484881 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61d06f-0053-4e32-9dcd-6cb0e6231350 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92dc78f8532b441e9a37c74c6a15ef18)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:37.486824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd61d06f-0053-4e32-9dcd-6cb0e6231350 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:37.511351 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1113/2228] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:37.515775 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f85846-6793-4421-93ed-99d9897a6906 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:37.536314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f85846-6793-4421-93ed-99d9897a6906 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:37.541346 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327d4a0-75c6-4675-bc63-45538b68094b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:37.542277 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327d4a0-75c6-4675-bc63-45538b68094b admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:37.542575 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327d4a0-75c6-4675-bc63-45538b68094b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:37.545825 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327d4a0-75c6-4675-bc63-45538b68094b admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:37.545825 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a327d4a0-75c6-4675-bc63-45538b68094b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:35:37.545825 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a327d4a0-75c6-4675-bc63-45538b68094b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:37.545825 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a327d4a0-75c6-4675-bc63-45538b68094b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:37.557972 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1114/2229] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:35:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:35:37.562271 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327d4a0-75c6-4675-bc63-45538b68094b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:37.572497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327d4a0-75c6-4675-bc63-45538b68094b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:37.578115 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31b4d6-100a-41b2-80e0-6e1a131c888a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:37.578774 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31b4d6-100a-41b2-80e0-6e1a131c888a admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:37.579066 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31b4d6-100a-41b2-80e0-6e1a131c888a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:37.579379 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a31b4d6-100a-41b2-80e0-6e1a131c888a admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:37.580078 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31b4d6-100a-41b2-80e0-6e1a131c888a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:37.582060 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a31b4d6-100a-41b2-80e0-6e1a131c888a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:37.643735 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bd61d06f-0053-4e32-9dcd-6cb0e6231350 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92dc78f8532b441e9a37c74c6a15ef18', 'request_id': 'req-bd61d06f-0053-4e32-9dcd-6cb0e6231350'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:37.650091 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bd61d06f-0053-4e32-9dcd-6cb0e6231350 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92dc78f8532b441e9a37c74c6a15ef18', 'request_id': 'req-bd61d06f-0053-4e32-9dcd-6cb0e6231350'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:37.656627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bd61d06f-0053-4e32-9dcd-6cb0e6231350 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92dc78f8532b441e9a37c74c6a15ef18', 'request_id': 'req-bd61d06f-0053-4e32-9dcd-6cb0e6231350'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:37.718298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1115/2230] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:35:37 2022] POST /identity/v3/projects => generated 377 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:37.719227 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31b4d6-100a-41b2-80e0-6e1a131c888a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:37.734616 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bd61d06f-0053-4e32-9dcd-6cb0e6231350 admin admin] The token cache is being invalidate because project 92dc78f8532b441e9a37c74c6a15ef18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:35:37.734912 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bd61d06f-0053-4e32-9dcd-6cb0e6231350 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:37.735643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bd61d06f-0053-4e32-9dcd-6cb0e6231350 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:37.742192 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1116/2231] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:35:37 2022] DELETE /identity/v3/projects/92dc78f8532b441e9a37c74c6a15ef18 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:35:37.744352 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5986c-2263-45a1-92cd-821f7013c500 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:37.744649 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5986c-2263-45a1-92cd-821f7013c500 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:37.745418 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d5986c-2263-45a1-92cd-821f7013c500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:37.767791 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-85d5986c-2263-45a1-92cd-821f7013c500 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:37.768802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-85d5986c-2263-45a1-92cd-821f7013c500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:37.792666 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a31b4d6-100a-41b2-80e0-6e1a131c888a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:37.797611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607536ad-cabc-4472-8fa9-eda98a466274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:37.798390 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607536ad-cabc-4472-8fa9-eda98a466274 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:37.798654 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607536ad-cabc-4472-8fa9-eda98a466274 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:37.798945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607536ad-cabc-4472-8fa9-eda98a466274 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:37.799678 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607536ad-cabc-4472-8fa9-eda98a466274 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:37.801628 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607536ad-cabc-4472-8fa9-eda98a466274 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:37.833098 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1117/2232] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:37.841282 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fa213-5b58-40ac-8fee-da4c7dfcf557 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:37.841577 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fa213-5b58-40ac-8fee-da4c7dfcf557 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:37.841859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728fa213-5b58-40ac-8fee-da4c7dfcf557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:37.859827 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-728fa213-5b58-40ac-8fee-da4c7dfcf557 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:37.861065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-728fa213-5b58-40ac-8fee-da4c7dfcf557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:37.870932 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1118/2233] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:35:37.873986 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728fa213-5b58-40ac-8fee-da4c7dfcf557 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:37.884047 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1116/2234] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:35:37 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:37.885293 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728fa213-5b58-40ac-8fee-da4c7dfcf557 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:37.888605 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607536ad-cabc-4472-8fa9-eda98a466274 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:37.891518 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-607536ad-cabc-4472-8fa9-eda98a466274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:37.893584 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c8c1a-cf1f-426f-869f-2f7094b6ce08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:37.894318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c8c1a-cf1f-426f-869f-2f7094b6ce08 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:37.894610 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c8c1a-cf1f-426f-869f-2f7094b6ce08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:37.894881 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71c8c1a-cf1f-426f-869f-2f7094b6ce08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:37.895685 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c8c1a-cf1f-426f-869f-2f7094b6ce08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:35:37.899760 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c8c1a-cf1f-426f-869f-2f7094b6ce08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:37.899760 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71c8c1a-cf1f-426f-869f-2f7094b6ce08 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:37.906743 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607536ad-cabc-4472-8fa9-eda98a466274 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:37.910598 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1119/2235] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:35:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:35:37.914829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c8c1a-cf1f-426f-869f-2f7094b6ce08 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:37.915534 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f755a-e695-4ef0-ab1e-a553066d3dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:37.916221 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f755a-e695-4ef0-ab1e-a553066d3dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:37.916506 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f755a-e695-4ef0-ab1e-a553066d3dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:37.917581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236f755a-e695-4ef0-ab1e-a553066d3dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:37.918286 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f755a-e695-4ef0-ab1e-a553066d3dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:37.920057 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236f755a-e695-4ef0-ab1e-a553066d3dd4 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:37.925968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71c8c1a-cf1f-426f-869f-2f7094b6ce08 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:37.935097 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1117/2236] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:35:37.937022 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2927152-7f58-4848-a67f-68f0abb81cfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:37.937750 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2927152-7f58-4848-a67f-68f0abb81cfd admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:37.938035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2927152-7f58-4848-a67f-68f0abb81cfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:37.938337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2927152-7f58-4848-a67f-68f0abb81cfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:37.939038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2927152-7f58-4848-a67f-68f0abb81cfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:37.939143 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f755a-e695-4ef0-ab1e-a553066d3dd4 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:37.941045 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2927152-7f58-4848-a67f-68f0abb81cfd admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:37.951596 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236f755a-e695-4ef0-ab1e-a553066d3dd4 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:37.956995 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e30166-47aa-4049-82b2-6afe99509a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:37.957795 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e30166-47aa-4049-82b2-6afe99509a21 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:37.958317 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e30166-47aa-4049-82b2-6afe99509a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:37.958707 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e30166-47aa-4049-82b2-6afe99509a21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:37.959646 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e30166-47aa-4049-82b2-6afe99509a21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:37.961494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e30166-47aa-4049-82b2-6afe99509a21 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:37.973497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1118/2237] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:35:37.977571 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e30166-47aa-4049-82b2-6afe99509a21 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:37.987572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e30166-47aa-4049-82b2-6afe99509a21 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:37.992462 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b89d48-88b6-48d6-8fad-8283c4764398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:37.993398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b89d48-88b6-48d6-8fad-8283c4764398 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:37.993684 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b89d48-88b6-48d6-8fad-8283c4764398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:37.993965 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b89d48-88b6-48d6-8fad-8283c4764398 admin admin] PATH_INFO: `/v3/projects/9ccf8704afeb40998257499b7b0a02c2/users/7609e864afe84047a9d3f4bb7ae99e60/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:38.012121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b89d48-88b6-48d6-8fad-8283c4764398 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ccf8704afeb40998257499b7b0a02c2, user_id=7609e864afe84047a9d3f4bb7ae99e60, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:38.014988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b89d48-88b6-48d6-8fad-8283c4764398 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:38.094370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1120/2238] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:35:37 2022] POST /identity/v3/projects => generated 365 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:35:38.095571 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2927152-7f58-4848-a67f-68f0abb81cfd admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:38.106529 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1119/2239] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:35:37 2022] PUT /identity/v3/projects/9ccf8704afeb40998257499b7b0a02c2/users/7609e864afe84047a9d3f4bb7ae99e60/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:35:38.116965 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7f99c-b544-4237-beec-132f1aa29dfd None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:38.117351 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7f99c-b544-4237-beec-132f1aa29dfd None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:38.117641 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe7f99c-b544-4237-beec-132f1aa29dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:38.160464 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-cbe7f99c-b544-4237-beec-132f1aa29dfd None None] MFA Rules not processed for user `7609e864afe84047a9d3f4bb7ae99e60`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:38.162848 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cbe7f99c-b544-4237-beec-132f1aa29dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:38.197099 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2927152-7f58-4848-a67f-68f0abb81cfd admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:38.280185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b753750-01b8-42d1-8814-805e8eca4b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:38.281139 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b753750-01b8-42d1-8814-805e8eca4b2f admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:38.281768 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b753750-01b8-42d1-8814-805e8eca4b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:38.281768 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b753750-01b8-42d1-8814-805e8eca4b2f admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:38.282388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b753750-01b8-42d1-8814-805e8eca4b2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:38.285133 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b753750-01b8-42d1-8814-805e8eca4b2f admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:38.308728 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1120/2240] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:38.323331 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7f99c-b544-4237-beec-132f1aa29dfd None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:38.323331 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-cbe7f99c-b544-4237-beec-132f1aa29dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:38.377287 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1121/2241] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:35:38 2022] POST /identity/v3/users => generated 283 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:35:38.378341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b753750-01b8-42d1-8814-805e8eca4b2f admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:38.389766 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe7f99c-b544-4237-beec-132f1aa29dfd None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:38.393963 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b753750-01b8-42d1-8814-805e8eca4b2f admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:38.394165 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eddc61-23c3-4a65-978c-0787c132abea service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7a61d85908d42698b27b9fc00daa832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:38.395367 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eddc61-23c3-4a65-978c-0787c132abea service glance] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:38.396140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eddc61-23c3-4a65-978c-0787c132abea service glance] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:38.396140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1eddc61-23c3-4a65-978c-0787c132abea service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:38.401789 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e6c48-899a-49c8-9d68-c4711f3dfbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:38.403196 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e6c48-899a-49c8-9d68-c4711f3dfbc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:38.403653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e6c48-899a-49c8-9d68-c4711f3dfbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:38.406323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e6c48-899a-49c8-9d68-c4711f3dfbc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:38.406323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e6c48-899a-49c8-9d68-c4711f3dfbc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:38.406707 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eddc61-23c3-4a65-978c-0787c132abea service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:38.408108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e6c48-899a-49c8-9d68-c4711f3dfbc8 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:38.408228 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1eddc61-23c3-4a65-978c-0787c132abea service glance] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:38.418562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1121/2242] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:38 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:35:38.420945 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1122/2243] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:35:38.424693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eddc61-23c3-4a65-978c-0787c132abea service glance] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:38.439678 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1eddc61-23c3-4a65-978c-0787c132abea service glance] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:38.450224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090374f5-6b28-4bc6-8eb3-2eba2f48b3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:38.451021 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090374f5-6b28-4bc6-8eb3-2eba2f48b3c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:38.451578 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090374f5-6b28-4bc6-8eb3-2eba2f48b3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:38.451953 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090374f5-6b28-4bc6-8eb3-2eba2f48b3c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:38.452964 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090374f5-6b28-4bc6-8eb3-2eba2f48b3c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:38.456816 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090374f5-6b28-4bc6-8eb3-2eba2f48b3c0 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:38.474767 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1122/2244] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:35:38.475853 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e6c48-899a-49c8-9d68-c4711f3dfbc8 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:38.495927 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e6c48-899a-49c8-9d68-c4711f3dfbc8 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:38.514640 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993208e6-ed8b-4eef-91a9-869c5cea2fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:38.516353 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993208e6-ed8b-4eef-91a9-869c5cea2fcc admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:38.516644 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993208e6-ed8b-4eef-91a9-869c5cea2fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:38.516934 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993208e6-ed8b-4eef-91a9-869c5cea2fcc admin admin] PATH_INFO: `/v3/projects/77b41d8f0e4d4eadbc5947357d2d0c44/users/6822ff27450e4218b2811f2cf2d41999/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:38.552098 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993208e6-ed8b-4eef-91a9-869c5cea2fcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=77b41d8f0e4d4eadbc5947357d2d0c44, user_id=6822ff27450e4218b2811f2cf2d41999, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:38.554858 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993208e6-ed8b-4eef-91a9-869c5cea2fcc admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:38.652950 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1123/2245] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:35:38 2022] PUT /identity/v3/projects/77b41d8f0e4d4eadbc5947357d2d0c44/users/6822ff27450e4218b2811f2cf2d41999/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:38.664353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb4a7e-0e9b-4b1e-b54d-edf49c972277 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:38.664648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb4a7e-0e9b-4b1e-b54d-edf49c972277 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:38.664927 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effb4a7e-0e9b-4b1e-b54d-edf49c972277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:38.702404 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-effb4a7e-0e9b-4b1e-b54d-edf49c972277 None None] MFA Rules not processed for user `6822ff27450e4218b2811f2cf2d41999`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:38.705880 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-effb4a7e-0e9b-4b1e-b54d-edf49c972277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:38.765118 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:38.786228 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:38.818140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1123/2246] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:38 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:38.896035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993208e6-ed8b-4eef-91a9-869c5cea2fcc admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:38.898688 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-993208e6-ed8b-4eef-91a9-869c5cea2fcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:38.958652 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993208e6-ed8b-4eef-91a9-869c5cea2fcc admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:38.962338 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571246fc-998a-44f8-82dd-75cd2908d47f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:38.963028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571246fc-998a-44f8-82dd-75cd2908d47f service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:38.963348 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571246fc-998a-44f8-82dd-75cd2908d47f service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:38.963733 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571246fc-998a-44f8-82dd-75cd2908d47f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:38.975302 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571246fc-998a-44f8-82dd-75cd2908d47f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:38.979710 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571246fc-998a-44f8-82dd-75cd2908d47f service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:38.988630 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1124/2247] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:38 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:35:39.559650 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ff030-bb96-4351-8ccc-25e7a334abf3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:39.561591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ff030-bb96-4351-8ccc-25e7a334abf3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:39.561591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59ff030-bb96-4351-8ccc-25e7a334abf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:39.580713 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a59ff030-bb96-4351-8ccc-25e7a334abf3 None None] MFA Rules not processed for user `797a7b1e128f4102bac6430a60d2c0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:39.582042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a59ff030-bb96-4351-8ccc-25e7a334abf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:39.657160 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1124/2248] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:39.848360 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:40.406177 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571246fc-998a-44f8-82dd-75cd2908d47f service cinder] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:40.407728 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-571246fc-998a-44f8-82dd-75cd2908d47f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:40.432881 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:40.489601 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571246fc-998a-44f8-82dd-75cd2908d47f service cinder] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:40.490835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29763c-249e-4e25-9ae0-656205db2193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:40.492512 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29763c-249e-4e25-9ae0-656205db2193 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:40.492872 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29763c-249e-4e25-9ae0-656205db2193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:40.497219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe29763c-249e-4e25-9ae0-656205db2193 admin admin] PATH_INFO: `/v3/users/98fd19fec73e4a368b00049588dcc4a6` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:40.497219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29763c-249e-4e25-9ae0-656205db2193 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98fd19fec73e4a368b00049588dcc4a6)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:40.497465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe29763c-249e-4e25-9ae0-656205db2193 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:40.747079 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-fe29763c-249e-4e25-9ae0-656205db2193 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98fd19fec73e4a368b00049588dcc4a6'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:40.819722 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-fe29763c-249e-4e25-9ae0-656205db2193 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98fd19fec73e4a368b00049588dcc4a6'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:40.842890 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-fe29763c-249e-4e25-9ae0-656205db2193 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98fd19fec73e4a368b00049588dcc4a6'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:40.844034 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-fe29763c-249e-4e25-9ae0-656205db2193 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98fd19fec73e4a368b00049588dcc4a6'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:40.844739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-fe29763c-249e-4e25-9ae0-656205db2193 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98fd19fec73e4a368b00049588dcc4a6'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:40.867355 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1125/2249] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:35:40 2022] DELETE /identity/v3/users/98fd19fec73e4a368b00049588dcc4a6 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:40.870499 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c98b9-889b-4604-8a1c-0cf1259ad7c2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:40.871048 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c98b9-889b-4604-8a1c-0cf1259ad7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:40.871048 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c98b9-889b-4604-8a1c-0cf1259ad7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:40.894473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c14c98b9-889b-4604-8a1c-0cf1259ad7c2 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:40.896044 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c14c98b9-889b-4604-8a1c-0cf1259ad7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:40.996989 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1125/2250] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:41.013245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29763c-249e-4e25-9ae0-656205db2193 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:41.013245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-fe29763c-249e-4e25-9ae0-656205db2193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:41.198805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe29763c-249e-4e25-9ae0-656205db2193 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:41.205636 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b6efd-4987-4724-afd1-c437280df8a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:41.206325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b6efd-4987-4724-afd1-c437280df8a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:41.206640 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b6efd-4987-4724-afd1-c437280df8a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:41.206900 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b6efd-4987-4724-afd1-c437280df8a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:41.230376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b6efd-4987-4724-afd1-c437280df8a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:41.232852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b6efd-4987-4724-afd1-c437280df8a2 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:41.262944 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1126/2251] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:35:41.820782 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c98b9-889b-4604-8a1c-0cf1259ad7c2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:41.824258 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c14c98b9-889b-4604-8a1c-0cf1259ad7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:41.846058 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c98b9-889b-4604-8a1c-0cf1259ad7c2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:41.864847 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd4eacb-f313-4d2d-82dc-a7e82b1397f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:41.867222 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4eacb-f313-4d2d-82dc-a7e82b1397f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:41.868007 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4eacb-f313-4d2d-82dc-a7e82b1397f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:41.868678 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd4eacb-f313-4d2d-82dc-a7e82b1397f8 admin admin] PATH_INFO: `/v3/projects/c6b045f359b149c4aab039644ee02a61` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:41.871490 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd4eacb-f313-4d2d-82dc-a7e82b1397f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6b045f359b149c4aab039644ee02a61)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:41.877211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd4eacb-f313-4d2d-82dc-a7e82b1397f8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:42.019856 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5cd4eacb-f313-4d2d-82dc-a7e82b1397f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6b045f359b149c4aab039644ee02a61', 'request_id': 'req-5cd4eacb-f313-4d2d-82dc-a7e82b1397f8'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:42.161903 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5cd4eacb-f313-4d2d-82dc-a7e82b1397f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6b045f359b149c4aab039644ee02a61', 'request_id': 'req-5cd4eacb-f313-4d2d-82dc-a7e82b1397f8'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:42.169035 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5cd4eacb-f313-4d2d-82dc-a7e82b1397f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6b045f359b149c4aab039644ee02a61', 'request_id': 'req-5cd4eacb-f313-4d2d-82dc-a7e82b1397f8'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:42.180711 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5cd4eacb-f313-4d2d-82dc-a7e82b1397f8 admin admin] The token cache is being invalidate because project c6b045f359b149c4aab039644ee02a61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:35:42.181105 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5cd4eacb-f313-4d2d-82dc-a7e82b1397f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:42.181829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5cd4eacb-f313-4d2d-82dc-a7e82b1397f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:35:42.183622 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1126/2252] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:35:41 2022] DELETE /identity/v3/projects/c6b045f359b149c4aab039644ee02a61 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:42.189031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7a749-594f-4411-af64-88f671fce64a None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:42.189548 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7a749-594f-4411-af64-88f671fce64a None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:42.189633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7a749-594f-4411-af64-88f671fce64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:42.208313 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-d4b7a749-594f-4411-af64-88f671fce64a None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:42.209514 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d4b7a749-594f-4411-af64-88f671fce64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:42.244819 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:42.270922 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1127/2253] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:35:42.280851 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965cf94-1bf0-4011-82d8-e7ba07d5d945 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:42.281347 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965cf94-1bf0-4011-82d8-e7ba07d5d945 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:42.281702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5965cf94-1bf0-4011-82d8-e7ba07d5d945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:42.302729 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5965cf94-1bf0-4011-82d8-e7ba07d5d945 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:42.303969 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5965cf94-1bf0-4011-82d8-e7ba07d5d945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:42.320872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1127/2254] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:42.321685 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7a749-594f-4411-af64-88f671fce64a None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:42.332239 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7a749-594f-4411-af64-88f671fce64a None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:42.336899 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287e8999-6aeb-4322-8734-bc8de1c5408c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:42.337558 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e8999-6aeb-4322-8734-bc8de1c5408c admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:42.337843 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e8999-6aeb-4322-8734-bc8de1c5408c admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:42.338118 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287e8999-6aeb-4322-8734-bc8de1c5408c admin admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:42.340355 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287e8999-6aeb-4322-8734-bc8de1c5408c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:35:42.340437 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287e8999-6aeb-4322-8734-bc8de1c5408c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:42.342022 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287e8999-6aeb-4322-8734-bc8de1c5408c admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:42.354038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1128/2255] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:35:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:35:42.358573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965cf94-1bf0-4011-82d8-e7ba07d5d945 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:42.374995 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5965cf94-1bf0-4011-82d8-e7ba07d5d945 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:42.381757 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3659ed-6801-409b-ae4d-5a1af6d2f0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:42.382580 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3659ed-6801-409b-ae4d-5a1af6d2f0be admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:42.389219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3659ed-6801-409b-ae4d-5a1af6d2f0be admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:42.389219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3659ed-6801-409b-ae4d-5a1af6d2f0be admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:42.389219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3659ed-6801-409b-ae4d-5a1af6d2f0be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:42.389219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3659ed-6801-409b-ae4d-5a1af6d2f0be admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:42.476194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1128/2256] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:35:42 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:42.477441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287e8999-6aeb-4322-8734-bc8de1c5408c admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:42.487755 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287e8999-6aeb-4322-8734-bc8de1c5408c admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:42.544501 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f47f3-e58e-44c3-b6c6-fcfd12b498e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:42.545168 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f47f3-e58e-44c3-b6c6-fcfd12b498e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:42.545452 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f47f3-e58e-44c3-b6c6-fcfd12b498e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:42.545727 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f47f3-e58e-44c3-b6c6-fcfd12b498e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:42.546406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65f47f3-e58e-44c3-b6c6-fcfd12b498e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:42.548345 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65f47f3-e58e-44c3-b6c6-fcfd12b498e7 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:42.656892 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3659ed-6801-409b-ae4d-5a1af6d2f0be admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:42.657918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1129/2257] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:35:42 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:42.676061 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3659ed-6801-409b-ae4d-5a1af6d2f0be admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:42.682011 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e95c2-dd22-470e-9680-79c2b7816db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:42.683355 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e95c2-dd22-470e-9680-79c2b7816db2 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:42.683851 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e95c2-dd22-470e-9680-79c2b7816db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:42.684471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e95c2-dd22-470e-9680-79c2b7816db2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:42.685714 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e95c2-dd22-470e-9680-79c2b7816db2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:42.688408 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e95c2-dd22-470e-9680-79c2b7816db2 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:42.700966 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1129/2258] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:35:42.717527 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f47f3-e58e-44c3-b6c6-fcfd12b498e7 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:42.728411 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f47f3-e58e-44c3-b6c6-fcfd12b498e7 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:42.734019 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea55b3e4-ad81-449d-838f-db0532f6ad0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:42.734692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55b3e4-ad81-449d-838f-db0532f6ad0b admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:42.734980 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55b3e4-ad81-449d-838f-db0532f6ad0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:42.735297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55b3e4-ad81-449d-838f-db0532f6ad0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:42.737444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea55b3e4-ad81-449d-838f-db0532f6ad0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:42.738603 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea55b3e4-ad81-449d-838f-db0532f6ad0b admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:42.756184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1130/2259] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:35:42.760677 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e95c2-dd22-470e-9680-79c2b7816db2 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:42.801937 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e95c2-dd22-470e-9680-79c2b7816db2 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:42.808340 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9716597-df63-4b60-bc47-75e03b198ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:42.809343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9716597-df63-4b60-bc47-75e03b198ff9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:42.809707 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9716597-df63-4b60-bc47-75e03b198ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:42.810063 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9716597-df63-4b60-bc47-75e03b198ff9 admin admin] PATH_INFO: `/v3/projects/ff8c3b3c0caa43d4b75ae94fc89c5568/users/c9826b2f5d424025b8daff42e066419a/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:42.816884 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:42.840546 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9716597-df63-4b60-bc47-75e03b198ff9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff8c3b3c0caa43d4b75ae94fc89c5568, user_id=c9826b2f5d424025b8daff42e066419a, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:42.846306 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9716597-df63-4b60-bc47-75e03b198ff9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:42.960484 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1130/2260] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:35:42 2022] PUT /identity/v3/projects/ff8c3b3c0caa43d4b75ae94fc89c5568/users/c9826b2f5d424025b8daff42e066419a/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:42.966560 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613dd0ce-6669-42e6-9599-0a20409a4bcc None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:42.966873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613dd0ce-6669-42e6-9599-0a20409a4bcc None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:42.967195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613dd0ce-6669-42e6-9599-0a20409a4bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:43.000911 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-613dd0ce-6669-42e6-9599-0a20409a4bcc None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:43.002157 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-613dd0ce-6669-42e6-9599-0a20409a4bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:43.086408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1131/2261] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:43.095403 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9716597-df63-4b60-bc47-75e03b198ff9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:43.105339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a9716597-df63-4b60-bc47-75e03b198ff9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:43.181072 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9716597-df63-4b60-bc47-75e03b198ff9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:43.187460 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8663c-1ddc-43a7-802d-d7a6a6cd3a26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:43.188125 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8663c-1ddc-43a7-802d-d7a6a6cd3a26 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:43.188416 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8663c-1ddc-43a7-802d-d7a6a6cd3a26 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:43.188701 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8663c-1ddc-43a7-802d-d7a6a6cd3a26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:43.198512 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8663c-1ddc-43a7-802d-d7a6a6cd3a26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:43.200032 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8663c-1ddc-43a7-802d-d7a6a6cd3a26 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:43.218649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1131/2262] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:35:44.406125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ffb7b-fbd1-4732-93c9-4d9b1f18db96 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:44.406626 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ffb7b-fbd1-4732-93c9-4d9b1f18db96 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:44.406717 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923ffb7b-fbd1-4732-93c9-4d9b1f18db96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:44.430197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-923ffb7b-fbd1-4732-93c9-4d9b1f18db96 None None] MFA Rules not processed for user `66ad8defb9b6453c96f1a8a7343695ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:44.432980 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-923ffb7b-fbd1-4732-93c9-4d9b1f18db96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:44.492588 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:44.504076 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1132/2263] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jun 30 10:35:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:35:44.550447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8663c-1ddc-43a7-802d-d7a6a6cd3a26 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:44.551942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f4e8663c-1ddc-43a7-802d-d7a6a6cd3a26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:44.637200 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8663c-1ddc-43a7-802d-d7a6a6cd3a26 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:44.642141 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34765d1-a3fa-4d43-abf1-757be29727dd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07de926cb8594b098f79111dab8a9cc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:44.642813 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34765d1-a3fa-4d43-abf1-757be29727dd service swift] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:44.643094 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34765d1-a3fa-4d43-abf1-757be29727dd service swift] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:44.643397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34765d1-a3fa-4d43-abf1-757be29727dd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:44.651665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34765d1-a3fa-4d43-abf1-757be29727dd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:44.653310 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34765d1-a3fa-4d43-abf1-757be29727dd service swift] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:44.662593 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1132/2264] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jun 30 10:35:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:35:50.840542 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a022a-0540-43f3-b78c-c4dc1cf8e4a9 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:50.841126 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a022a-0540-43f3-b78c-c4dc1cf8e4a9 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:50.841126 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343a022a-0540-43f3-b78c-c4dc1cf8e4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:50.878448 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-343a022a-0540-43f3-b78c-c4dc1cf8e4a9 None None] MFA Rules not processed for user `c9826b2f5d424025b8daff42e066419a`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:50.880169 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-343a022a-0540-43f3-b78c-c4dc1cf8e4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:50.984781 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1133/2265] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:50.986063 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34765d1-a3fa-4d43-abf1-757be29727dd service swift] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:51.001508 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34765d1-a3fa-4d43-abf1-757be29727dd service swift] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:51.006492 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67471e65-d665-4b03-a0d5-3652cbebad7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:51.007090 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67471e65-d665-4b03-a0d5-3652cbebad7d admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:51.007407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67471e65-d665-4b03-a0d5-3652cbebad7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:51.007688 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67471e65-d665-4b03-a0d5-3652cbebad7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:51.008538 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67471e65-d665-4b03-a0d5-3652cbebad7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:51.013529 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67471e65-d665-4b03-a0d5-3652cbebad7d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:51.114628 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1133/2266] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:35:50 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:51.119855 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a022a-0540-43f3-b78c-c4dc1cf8e4a9 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:51.131503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343a022a-0540-43f3-b78c-c4dc1cf8e4a9 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:51.189816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60a62f-e984-49cc-8678-1756b2f3c06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:51.190494 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60a62f-e984-49cc-8678-1756b2f3c06e admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:51.190781 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60a62f-e984-49cc-8678-1756b2f3c06e admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:51.191058 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c60a62f-e984-49cc-8678-1756b2f3c06e admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:51.191880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c60a62f-e984-49cc-8678-1756b2f3c06e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:51.194007 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c60a62f-e984-49cc-8678-1756b2f3c06e admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:51.298923 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1134/2267] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:35:51 2022] POST /identity/v3/users => generated 298 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:35:51.303508 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67471e65-d665-4b03-a0d5-3652cbebad7d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:51.316251 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67471e65-d665-4b03-a0d5-3652cbebad7d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:51.322113 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a81682-ad17-4186-b51d-654e8757792a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:51.323179 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a81682-ad17-4186-b51d-654e8757792a admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:51.323553 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a81682-ad17-4186-b51d-654e8757792a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:51.324051 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a81682-ad17-4186-b51d-654e8757792a admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:51.324893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a81682-ad17-4186-b51d-654e8757792a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:51.326958 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a81682-ad17-4186-b51d-654e8757792a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:51.339082 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1134/2268] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:35:51.343578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60a62f-e984-49cc-8678-1756b2f3c06e admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:51.355909 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c60a62f-e984-49cc-8678-1756b2f3c06e admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:51.361078 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb14f46-9cbc-4a19-b4b6-92a489f02311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:51.362156 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb14f46-9cbc-4a19-b4b6-92a489f02311 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:51.362444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb14f46-9cbc-4a19-b4b6-92a489f02311 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:51.362722 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb14f46-9cbc-4a19-b4b6-92a489f02311 admin admin] PATH_INFO: `/v3/projects/4d0dd3b46456410183c7a1f23069e964/users/ef14698bfec847039304f3a016238059/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:51.378897 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb14f46-9cbc-4a19-b4b6-92a489f02311 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d0dd3b46456410183c7a1f23069e964, user_id=ef14698bfec847039304f3a016238059, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:51.381608 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb14f46-9cbc-4a19-b4b6-92a489f02311 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:51.505695 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1135/2269] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:35:51 2022] PUT /identity/v3/projects/4d0dd3b46456410183c7a1f23069e964/users/ef14698bfec847039304f3a016238059/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:51.509589 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a81682-ad17-4186-b51d-654e8757792a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:51.520175 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a81682-ad17-4186-b51d-654e8757792a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:51.565424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:51.581208 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27bf54f-1266-49c4-97ee-81f10adfc62e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:51.581880 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27bf54f-1266-49c4-97ee-81f10adfc62e admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:51.582166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27bf54f-1266-49c4-97ee-81f10adfc62e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:51.582450 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27bf54f-1266-49c4-97ee-81f10adfc62e admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:51.583177 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27bf54f-1266-49c4-97ee-81f10adfc62e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:51.584895 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27bf54f-1266-49c4-97ee-81f10adfc62e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:51.602333 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1135/2270] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:35:51.618142 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb14f46-9cbc-4a19-b4b6-92a489f02311 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:51.631305 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb14f46-9cbc-4a19-b4b6-92a489f02311 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:51.636558 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9959e382-fd5c-4101-a7d1-c3d1a0a8c8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:51.637511 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9959e382-fd5c-4101-a7d1-c3d1a0a8c8df admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:51.637797 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9959e382-fd5c-4101-a7d1-c3d1a0a8c8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:51.638072 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9959e382-fd5c-4101-a7d1-c3d1a0a8c8df admin admin] PATH_INFO: `/v3/projects/4d0dd3b46456410183c7a1f23069e964/users/ef14698bfec847039304f3a016238059/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:51.640326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9959e382-fd5c-4101-a7d1-c3d1a0a8c8df admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d0dd3b46456410183c7a1f23069e964, user_id=ef14698bfec847039304f3a016238059, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:51.643027 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9959e382-fd5c-4101-a7d1-c3d1a0a8c8df admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:51.654958 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1136/2271] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:35:51 2022] PUT /identity/v3/projects/4d0dd3b46456410183c7a1f23069e964/users/ef14698bfec847039304f3a016238059/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:35:56.480697 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f452019-6c7c-452c-a552-f33e5c27fdca None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:56.481161 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f452019-6c7c-452c-a552-f33e5c27fdca None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:56.481391 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f452019-6c7c-452c-a552-f33e5c27fdca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:56.541319 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4f452019-6c7c-452c-a552-f33e5c27fdca None None] MFA Rules not processed for user `66ad8defb9b6453c96f1a8a7343695ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:56.542971 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4f452019-6c7c-452c-a552-f33e5c27fdca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:56.646470 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1136/2272] 10.222.0.47 () {58 vars in 967 bytes} [Thu Jun 30 10:35:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:35:56.660434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9959e382-fd5c-4101-a7d1-c3d1a0a8c8df admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:56.707598 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:35:56.746633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9959e382-fd5c-4101-a7d1-c3d1a0a8c8df admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:56.756029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0737cb0-913a-43cb-aad4-fb17d4e38679 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07de926cb8594b098f79111dab8a9cc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:56.760571 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0737cb0-913a-43cb-aad4-fb17d4e38679 service swift] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:56.760571 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0737cb0-913a-43cb-aad4-fb17d4e38679 service swift] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:56.760571 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0737cb0-913a-43cb-aad4-fb17d4e38679 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:56.779538 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0737cb0-913a-43cb-aad4-fb17d4e38679 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:56.783471 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0737cb0-913a-43cb-aad4-fb17d4e38679 service swift] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:56.802546 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1137/2273] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jun 30 10:35:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:35:57.834280 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a76965-d5dc-4590-a283-23bb6d80dc51 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:57.836897 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a76965-d5dc-4590-a283-23bb6d80dc51 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:57.837205 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a76965-d5dc-4590-a283-23bb6d80dc51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:57.858916 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-58a76965-d5dc-4590-a283-23bb6d80dc51 None None] MFA Rules not processed for user `66ad8defb9b6453c96f1a8a7343695ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:57.860476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-58a76965-d5dc-4590-a283-23bb6d80dc51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:57.881850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1137/2274] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jun 30 10:35:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:57.895686 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0737cb0-913a-43cb-aad4-fb17d4e38679 service swift] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:57.909765 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0737cb0-913a-43cb-aad4-fb17d4e38679 service swift] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:57.916655 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4889bc1-3b69-4edb-b061-a17f30054e06 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07de926cb8594b098f79111dab8a9cc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:57.918072 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4889bc1-3b69-4edb-b061-a17f30054e06 service swift] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:57.918459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4889bc1-3b69-4edb-b061-a17f30054e06 service swift] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:57.918803 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4889bc1-3b69-4edb-b061-a17f30054e06 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:57.929743 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4889bc1-3b69-4edb-b061-a17f30054e06 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:57.935446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4889bc1-3b69-4edb-b061-a17f30054e06 service swift] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:57.959339 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1138/2275] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jun 30 10:35:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:35:58.947745 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd72fc2-2a27-49ca-b01b-2c41bf2b00cb None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:58.948979 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd72fc2-2a27-49ca-b01b-2c41bf2b00cb None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:58.949448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd72fc2-2a27-49ca-b01b-2c41bf2b00cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:58.978084 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ebd72fc2-2a27-49ca-b01b-2c41bf2b00cb None None] MFA Rules not processed for user `66ad8defb9b6453c96f1a8a7343695ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:58.980480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ebd72fc2-2a27-49ca-b01b-2c41bf2b00cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:59.011697 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1138/2276] 10.222.0.47 () {58 vars in 967 bytes} [Thu Jun 30 10:35:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:35:59.034065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4889bc1-3b69-4edb-b061-a17f30054e06 service swift] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:59.049290 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4889bc1-3b69-4edb-b061-a17f30054e06 service swift] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:59.056346 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d795e2-a0a3-4761-a4c4-4cc91fb8840c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07de926cb8594b098f79111dab8a9cc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:59.057129 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d795e2-a0a3-4761-a4c4-4cc91fb8840c service swift] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:59.057591 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d795e2-a0a3-4761-a4c4-4cc91fb8840c service swift] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:59.058096 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d795e2-a0a3-4761-a4c4-4cc91fb8840c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:59.068885 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ceef0-049d-4c09-a295-a730cfd23735 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:59.069173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ceef0-049d-4c09-a295-a730cfd23735 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:59.069451 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348ceef0-049d-4c09-a295-a730cfd23735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:59.079191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d795e2-a0a3-4761-a4c4-4cc91fb8840c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:59.082792 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d795e2-a0a3-4761-a4c4-4cc91fb8840c service swift] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:59.099354 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1139/2277] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jun 30 10:35:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:35:59.117374 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-348ceef0-049d-4c09-a295-a730cfd23735 None None] MFA Rules not processed for user `ef14698bfec847039304f3a016238059`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:35:59.117374 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-348ceef0-049d-4c09-a295-a730cfd23735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:59.239100 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1139/2278] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:35:59 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:35:59.544879 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d795e2-a0a3-4761-a4c4-4cc91fb8840c service swift] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:35:59.546309 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-c3d795e2-a0a3-4761-a4c4-4cc91fb8840c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:35:59.645546 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d795e2-a0a3-4761-a4c4-4cc91fb8840c service swift] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:35:59.649461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32419404-5f5c-493b-81a8-133912ff7337 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7a61d85908d42698b27b9fc00daa832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:35:59.650114 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32419404-5f5c-493b-81a8-133912ff7337 service glance] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:35:59.650434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32419404-5f5c-493b-81a8-133912ff7337 service glance] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:35:59.650714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32419404-5f5c-493b-81a8-133912ff7337 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:35:59.723297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32419404-5f5c-493b-81a8-133912ff7337 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:35:59.724683 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32419404-5f5c-493b-81a8-133912ff7337 service glance] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:35:59.734438 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1140/2279] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:35:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:36:00.037672 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a32db5-505d-43f3-b272-9bfae047cb76 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:00.038496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a32db5-505d-43f3-b272-9bfae047cb76 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:00.038782 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a32db5-505d-43f3-b272-9bfae047cb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:00.069148 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-62a32db5-505d-43f3-b272-9bfae047cb76 None None] MFA Rules not processed for user `66ad8defb9b6453c96f1a8a7343695ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:36:00.070618 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-62a32db5-505d-43f3-b272-9bfae047cb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:00.082312 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1140/2280] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jun 30 10:36:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:36:00.098277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32419404-5f5c-493b-81a8-133912ff7337 service glance] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:00.137102 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32419404-5f5c-493b-81a8-133912ff7337 service glance] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:00.159283 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379b688-0d79-4191-9a5b-1ccf1975b673 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07de926cb8594b098f79111dab8a9cc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:00.160908 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379b688-0d79-4191-9a5b-1ccf1975b673 service swift] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:00.161209 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379b688-0d79-4191-9a5b-1ccf1975b673 service swift] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:00.161498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379b688-0d79-4191-9a5b-1ccf1975b673 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:00.177568 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2379b688-0d79-4191-9a5b-1ccf1975b673 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:00.179605 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2379b688-0d79-4191-9a5b-1ccf1975b673 service swift] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:00.196340 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1141/2281] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jun 30 10:36:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:36:02.546468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a32db5-505d-43f3-b272-9bfae047cb76 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:02.548510 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-62a32db5-505d-43f3-b272-9bfae047cb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:02.654816 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:36:02.722576 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a32db5-505d-43f3-b272-9bfae047cb76 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:02.728104 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819cd62-1829-427f-aa56-e6d0b4221640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:02.729378 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819cd62-1829-427f-aa56-e6d0b4221640 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:02.730005 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819cd62-1829-427f-aa56-e6d0b4221640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:02.730619 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819cd62-1829-427f-aa56-e6d0b4221640 admin admin] PATH_INFO: `/v3/users/c67926b0c8394ca28ebef50230387127` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:02.732740 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819cd62-1829-427f-aa56-e6d0b4221640 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c67926b0c8394ca28ebef50230387127)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:02.736235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819cd62-1829-427f-aa56-e6d0b4221640 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:02.959457 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8819cd62-1829-427f-aa56-e6d0b4221640 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c67926b0c8394ca28ebef50230387127'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:02.960812 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8819cd62-1829-427f-aa56-e6d0b4221640 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c67926b0c8394ca28ebef50230387127'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:02.976280 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8819cd62-1829-427f-aa56-e6d0b4221640 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c67926b0c8394ca28ebef50230387127'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:02.976968 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8819cd62-1829-427f-aa56-e6d0b4221640 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c67926b0c8394ca28ebef50230387127'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:03.093477 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8819cd62-1829-427f-aa56-e6d0b4221640 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c67926b0c8394ca28ebef50230387127'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:03.135909 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f448d4-93be-42dd-8b68-c4b1b741605c None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:03.136832 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f448d4-93be-42dd-8b68-c4b1b741605c None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:03.137326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f448d4-93be-42dd-8b68-c4b1b741605c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:03.139458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1141/2282] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:36:02 2022] DELETE /identity/v3/users/c67926b0c8394ca28ebef50230387127 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:36:03.165781 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-64f448d4-93be-42dd-8b68-c4b1b741605c None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:36:03.167884 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-64f448d4-93be-42dd-8b68-c4b1b741605c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:03.255109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:36:03.361846 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819cd62-1829-427f-aa56-e6d0b4221640 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:03.368356 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8819cd62-1829-427f-aa56-e6d0b4221640 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:03.375720 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1142/2283] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:36:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:36:03.478422 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819cd62-1829-427f-aa56-e6d0b4221640 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:03.489907 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f4dbc-379a-4844-94a6-e4b03bfd9267 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:03.490731 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f4dbc-379a-4844-94a6-e4b03bfd9267 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:03.492022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f4dbc-379a-4844-94a6-e4b03bfd9267 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:03.492358 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96f4dbc-379a-4844-94a6-e4b03bfd9267 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:03.521484 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f4dbc-379a-4844-94a6-e4b03bfd9267 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:03.526102 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96f4dbc-379a-4844-94a6-e4b03bfd9267 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:03.540112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1142/2284] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:36:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:36:03.913269 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f448d4-93be-42dd-8b68-c4b1b741605c None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:03.917748 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-64f448d4-93be-42dd-8b68-c4b1b741605c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:03.974038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f448d4-93be-42dd-8b68-c4b1b741605c None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:03.988682 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037f60b7-8b45-49d4-ad79-86faeea7b481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:03.997248 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f60b7-8b45-49d4-ad79-86faeea7b481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:03.998552 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f60b7-8b45-49d4-ad79-86faeea7b481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:03.999084 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037f60b7-8b45-49d4-ad79-86faeea7b481 admin admin] PATH_INFO: `/v3/projects/81540485036f472d9ae481a97dc32965` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:04.003375 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f60b7-8b45-49d4-ad79-86faeea7b481 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81540485036f472d9ae481a97dc32965)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:04.013725 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037f60b7-8b45-49d4-ad79-86faeea7b481 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:04.251178 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-037f60b7-8b45-49d4-ad79-86faeea7b481 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81540485036f472d9ae481a97dc32965', 'request_id': 'req-037f60b7-8b45-49d4-ad79-86faeea7b481'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:04.280075 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-037f60b7-8b45-49d4-ad79-86faeea7b481 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81540485036f472d9ae481a97dc32965', 'request_id': 'req-037f60b7-8b45-49d4-ad79-86faeea7b481'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:04.301963 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-037f60b7-8b45-49d4-ad79-86faeea7b481 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81540485036f472d9ae481a97dc32965', 'request_id': 'req-037f60b7-8b45-49d4-ad79-86faeea7b481'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:04.423549 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-037f60b7-8b45-49d4-ad79-86faeea7b481 admin admin] The token cache is being invalidate because project 81540485036f472d9ae481a97dc32965 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:36:04.423549 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-037f60b7-8b45-49d4-ad79-86faeea7b481 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:04.423549 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-037f60b7-8b45-49d4-ad79-86faeea7b481 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:04.427679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1143/2285] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:36:03 2022] DELETE /identity/v3/projects/81540485036f472d9ae481a97dc32965 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:36:10.896032 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f4dbc-379a-4844-94a6-e4b03bfd9267 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:10.897637 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b96f4dbc-379a-4844-94a6-e4b03bfd9267 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:10.923309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96f4dbc-379a-4844-94a6-e4b03bfd9267 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:11.061299 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:36:11.071585 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972608b8-42dd-4e13-934a-1353338b0600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:11.073750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972608b8-42dd-4e13-934a-1353338b0600 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:11.074065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972608b8-42dd-4e13-934a-1353338b0600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:11.074345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972608b8-42dd-4e13-934a-1353338b0600 admin admin] PATH_INFO: `/v3/users/7609e864afe84047a9d3f4bb7ae99e60` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:11.075720 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972608b8-42dd-4e13-934a-1353338b0600 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7609e864afe84047a9d3f4bb7ae99e60)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:11.077741 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972608b8-42dd-4e13-934a-1353338b0600 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:11.392040 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-972608b8-42dd-4e13-934a-1353338b0600 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7609e864afe84047a9d3f4bb7ae99e60'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:11.393025 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-972608b8-42dd-4e13-934a-1353338b0600 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7609e864afe84047a9d3f4bb7ae99e60'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:11.404844 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-972608b8-42dd-4e13-934a-1353338b0600 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7609e864afe84047a9d3f4bb7ae99e60'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:11.405783 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-972608b8-42dd-4e13-934a-1353338b0600 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7609e864afe84047a9d3f4bb7ae99e60'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:11.563575 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-972608b8-42dd-4e13-934a-1353338b0600 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7609e864afe84047a9d3f4bb7ae99e60'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:11.596389 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1143/2286] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:36:10 2022] DELETE /identity/v3/users/7609e864afe84047a9d3f4bb7ae99e60 => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:36:11.600494 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89247c-4b0e-4fd9-b42f-5b7049f23a44 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:11.601478 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89247c-4b0e-4fd9-b42f-5b7049f23a44 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:11.601973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89247c-4b0e-4fd9-b42f-5b7049f23a44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:11.635619 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-bc89247c-4b0e-4fd9-b42f-5b7049f23a44 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:36:11.636948 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-bc89247c-4b0e-4fd9-b42f-5b7049f23a44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:11.723896 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1144/2287] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:36:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:36:11.738434 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972608b8-42dd-4e13-934a-1353338b0600 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:11.740074 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-972608b8-42dd-4e13-934a-1353338b0600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:11.839526 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972608b8-42dd-4e13-934a-1353338b0600 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:11.846601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d2de2-5363-4a69-a8c8-9374497f855a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:11.847553 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d2de2-5363-4a69-a8c8-9374497f855a service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:11.847984 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d2de2-5363-4a69-a8c8-9374497f855a service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:11.848560 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d2de2-5363-4a69-a8c8-9374497f855a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:11.860880 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d2de2-5363-4a69-a8c8-9374497f855a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:11.862439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d2de2-5363-4a69-a8c8-9374497f855a service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:11.876583 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1144/2288] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:36:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:36:12.947983 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ed2a0-4424-490c-9886-5b2595239043 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:12.949215 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ed2a0-4424-490c-9886-5b2595239043 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:12.949464 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850ed2a0-4424-490c-9886-5b2595239043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:12.988530 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-850ed2a0-4424-490c-9886-5b2595239043 None None] MFA Rules not processed for user `eecc2db7b5754624b678e51dd9a37a46`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:36:12.988530 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-850ed2a0-4424-490c-9886-5b2595239043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:13.078898 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1145/2289] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:36:12 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:36:13.436168 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d2de2-5363-4a69-a8c8-9374497f855a service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:13.437784 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-530d2de2-5363-4a69-a8c8-9374497f855a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:13.453080 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d2de2-5363-4a69-a8c8-9374497f855a service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:13.458756 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e43a8-ca71-4a36-aa0c-1c4d47198812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:13.459824 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e43a8-ca71-4a36-aa0c-1c4d47198812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:13.459919 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e43a8-ca71-4a36-aa0c-1c4d47198812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:13.460247 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e43a8-ca71-4a36-aa0c-1c4d47198812 admin admin] PATH_INFO: `/v3/projects/9ccf8704afeb40998257499b7b0a02c2` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:13.461636 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e43a8-ca71-4a36-aa0c-1c4d47198812 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ccf8704afeb40998257499b7b0a02c2)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:13.464265 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e43a8-ca71-4a36-aa0c-1c4d47198812 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:13.633068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-207e43a8-ca71-4a36-aa0c-1c4d47198812 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ccf8704afeb40998257499b7b0a02c2', 'request_id': 'req-207e43a8-ca71-4a36-aa0c-1c4d47198812'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:13.721662 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-207e43a8-ca71-4a36-aa0c-1c4d47198812 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ccf8704afeb40998257499b7b0a02c2', 'request_id': 'req-207e43a8-ca71-4a36-aa0c-1c4d47198812'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:13.730122 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-207e43a8-ca71-4a36-aa0c-1c4d47198812 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ccf8704afeb40998257499b7b0a02c2', 'request_id': 'req-207e43a8-ca71-4a36-aa0c-1c4d47198812'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:13.747062 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-207e43a8-ca71-4a36-aa0c-1c4d47198812 admin admin] The token cache is being invalidate because project 9ccf8704afeb40998257499b7b0a02c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:36:13.748796 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-207e43a8-ca71-4a36-aa0c-1c4d47198812 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:13.748796 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-207e43a8-ca71-4a36-aa0c-1c4d47198812 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:13.763027 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1145/2290] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:36:13 2022] DELETE /identity/v3/projects/9ccf8704afeb40998257499b7b0a02c2 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:36:13.763994 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80e780-4f2a-4369-af57-d2e53953632c None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:13.764289 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80e780-4f2a-4369-af57-d2e53953632c None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:13.764574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d80e780-4f2a-4369-af57-d2e53953632c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:13.774484 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:36:13.803094 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-7d80e780-4f2a-4369-af57-d2e53953632c None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:36:13.804371 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-7d80e780-4f2a-4369-af57-d2e53953632c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:13.934457 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17773a0a-1438-424d-8a7d-ea6b469fa00c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:13.934652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17773a0a-1438-424d-8a7d-ea6b469fa00c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:13.934938 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17773a0a-1438-424d-8a7d-ea6b469fa00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:13.939007 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1146/2291] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:36:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:36:13.941885 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:36:13.956252 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-17773a0a-1438-424d-8a7d-ea6b469fa00c None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:36:13.957741 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-17773a0a-1438-424d-8a7d-ea6b469fa00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:13.967468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1146/2292] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:36:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:36:13.972796 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80e780-4f2a-4369-af57-d2e53953632c None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:13.984954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d80e780-4f2a-4369-af57-d2e53953632c None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:13.991097 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45fa3e6-c56f-4708-b84b-eff23919b964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:13.992319 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45fa3e6-c56f-4708-b84b-eff23919b964 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:13.992815 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45fa3e6-c56f-4708-b84b-eff23919b964 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:13.993463 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45fa3e6-c56f-4708-b84b-eff23919b964 admin admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:13.995218 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45fa3e6-c56f-4708-b84b-eff23919b964 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:36:13.995808 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45fa3e6-c56f-4708-b84b-eff23919b964 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:13.997682 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45fa3e6-c56f-4708-b84b-eff23919b964 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:14.049956 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1147/2293] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:36:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:36:14.056567 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17773a0a-1438-424d-8a7d-ea6b469fa00c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:14.067069 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17773a0a-1438-424d-8a7d-ea6b469fa00c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:14.072088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0a440-38e0-4e6c-a2fe-ebf25c58f0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:14.072752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0a440-38e0-4e6c-a2fe-ebf25c58f0d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:14.073040 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0a440-38e0-4e6c-a2fe-ebf25c58f0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:14.073322 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0a440-38e0-4e6c-a2fe-ebf25c58f0d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:14.074024 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0a440-38e0-4e6c-a2fe-ebf25c58f0d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:14.076552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0a440-38e0-4e6c-a2fe-ebf25c58f0d9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:14.188809 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1147/2294] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:36:14 2022] POST /identity/v3/projects => generated 373 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:36:14.192098 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45fa3e6-c56f-4708-b84b-eff23919b964 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:14.217040 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45fa3e6-c56f-4708-b84b-eff23919b964 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:14.286367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd9cd6-2785-4b83-80cf-73956de01be7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:14.286367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd9cd6-2785-4b83-80cf-73956de01be7 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:14.286367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd9cd6-2785-4b83-80cf-73956de01be7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:14.286367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cd9cd6-2785-4b83-80cf-73956de01be7 admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:14.286859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd9cd6-2785-4b83-80cf-73956de01be7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:14.286859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cd9cd6-2785-4b83-80cf-73956de01be7 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:14.441084 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1148/2295] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:36:14 2022] POST /identity/v3/users => generated 287 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:36:14.445224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0a440-38e0-4e6c-a2fe-ebf25c58f0d9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:14.459020 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0a440-38e0-4e6c-a2fe-ebf25c58f0d9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:14.464954 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59107835-0ef3-40d8-93a7-2736bab8afc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:14.465811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59107835-0ef3-40d8-93a7-2736bab8afc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:14.466180 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59107835-0ef3-40d8-93a7-2736bab8afc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:14.466636 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59107835-0ef3-40d8-93a7-2736bab8afc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:14.467790 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59107835-0ef3-40d8-93a7-2736bab8afc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:14.469610 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59107835-0ef3-40d8-93a7-2736bab8afc7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:14.481928 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1148/2296] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:36:14.486055 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd9cd6-2785-4b83-80cf-73956de01be7 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:14.496650 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cd9cd6-2785-4b83-80cf-73956de01be7 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:14.501436 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99838a7-550e-4e3f-bb8a-b7ee4379d5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:14.502166 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99838a7-550e-4e3f-bb8a-b7ee4379d5b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:14.502480 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99838a7-550e-4e3f-bb8a-b7ee4379d5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:14.503485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99838a7-550e-4e3f-bb8a-b7ee4379d5b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:14.504275 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99838a7-550e-4e3f-bb8a-b7ee4379d5b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:14.505889 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99838a7-550e-4e3f-bb8a-b7ee4379d5b9 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:14.518847 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1149/2297] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:36:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:36:14.522962 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59107835-0ef3-40d8-93a7-2736bab8afc7 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:14.534428 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59107835-0ef3-40d8-93a7-2736bab8afc7 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:14.538951 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbfee7-12d8-40bc-a344-a07c8609ce6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:14.539864 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbfee7-12d8-40bc-a344-a07c8609ce6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:14.540693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbfee7-12d8-40bc-a344-a07c8609ce6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:14.540693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcbfee7-12d8-40bc-a344-a07c8609ce6a admin admin] PATH_INFO: `/v3/projects/6295a17c5bf646fca12c5e55b0dd7bee/users/59256e514cd34b6eb230fca92b6b02c7/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:14.557178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcbfee7-12d8-40bc-a344-a07c8609ce6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6295a17c5bf646fca12c5e55b0dd7bee, user_id=59256e514cd34b6eb230fca92b6b02c7, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:14.560398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcbfee7-12d8-40bc-a344-a07c8609ce6a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:14.667037 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1149/2298] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:36:14 2022] PUT /identity/v3/projects/6295a17c5bf646fca12c5e55b0dd7bee/users/59256e514cd34b6eb230fca92b6b02c7/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:36:14.677347 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f46d96-3461-4cd7-8615-f84aa734e45d None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:14.677641 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f46d96-3461-4cd7-8615-f84aa734e45d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:14.677924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f46d96-3461-4cd7-8615-f84aa734e45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:14.704157 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-88f46d96-3461-4cd7-8615-f84aa734e45d None None] MFA Rules not processed for user `59256e514cd34b6eb230fca92b6b02c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:36:14.705306 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-88f46d96-3461-4cd7-8615-f84aa734e45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:14.777036 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:36:14.801094 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1150/2299] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:36:14.813642 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbfee7-12d8-40bc-a344-a07c8609ce6a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:14.815787 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6fcbfee7-12d8-40bc-a344-a07c8609ce6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:14.893370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fb0b8-45f5-46ab-bc24-8f61320fcffa None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:14.893570 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fb0b8-45f5-46ab-bc24-8f61320fcffa None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:14.893830 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293fb0b8-45f5-46ab-bc24-8f61320fcffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:14.912822 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcbfee7-12d8-40bc-a344-a07c8609ce6a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:14.916836 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fac192-91f4-44a1-93a0-aec0fe6109fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:14.917111 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-293fb0b8-45f5-46ab-bc24-8f61320fcffa None None] MFA Rules not processed for user `c9826b2f5d424025b8daff42e066419a`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:36:14.917872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fac192-91f4-44a1-93a0-aec0fe6109fc service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:14.918209 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fac192-91f4-44a1-93a0-aec0fe6109fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:14.918371 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-293fb0b8-45f5-46ab-bc24-8f61320fcffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:14.918486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fac192-91f4-44a1-93a0-aec0fe6109fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:14.928079 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fac192-91f4-44a1-93a0-aec0fe6109fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:14.929590 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fac192-91f4-44a1-93a0-aec0fe6109fc service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:14.940765 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1150/2300] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:36:14 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:36:14.976054 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fac192-91f4-44a1-93a0-aec0fe6109fc service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:14.988422 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1151/2301] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:36:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:36:15.064167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fac192-91f4-44a1-93a0-aec0fe6109fc service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:15.069335 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58edab60-7768-4656-8600-b17d0d1824e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:15.070218 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edab60-7768-4656-8600-b17d0d1824e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:15.070530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edab60-7768-4656-8600-b17d0d1824e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:15.070770 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58edab60-7768-4656-8600-b17d0d1824e8 admin admin] PATH_INFO: `/v3/users/59256e514cd34b6eb230fca92b6b02c7` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:15.073368 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58edab60-7768-4656-8600-b17d0d1824e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59256e514cd34b6eb230fca92b6b02c7)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:15.076417 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58edab60-7768-4656-8600-b17d0d1824e8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:15.128869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:36:15.284198 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-58edab60-7768-4656-8600-b17d0d1824e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59256e514cd34b6eb230fca92b6b02c7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:15.284949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-58edab60-7768-4656-8600-b17d0d1824e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59256e514cd34b6eb230fca92b6b02c7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:15.292710 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-58edab60-7768-4656-8600-b17d0d1824e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59256e514cd34b6eb230fca92b6b02c7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:15.293791 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-58edab60-7768-4656-8600-b17d0d1824e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59256e514cd34b6eb230fca92b6b02c7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:15.464826 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-58edab60-7768-4656-8600-b17d0d1824e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59256e514cd34b6eb230fca92b6b02c7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:15.486001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1151/2302] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:36:14 2022] DELETE /identity/v3/users/59256e514cd34b6eb230fca92b6b02c7 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:36:15.490509 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4878a-77e4-42ae-a1f0-a8911b989fa6 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:15.490804 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4878a-77e4-42ae-a1f0-a8911b989fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:15.491082 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b4878a-77e4-42ae-a1f0-a8911b989fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:15.510526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-57b4878a-77e4-42ae-a1f0-a8911b989fa6 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:36:15.511882 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-57b4878a-77e4-42ae-a1f0-a8911b989fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:15.593185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1152/2303] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:36:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:36:15.602655 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58edab60-7768-4656-8600-b17d0d1824e8 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:15.606038 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-58edab60-7768-4656-8600-b17d0d1824e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:15.679155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58edab60-7768-4656-8600-b17d0d1824e8 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:15.683620 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4d65c-a860-4d10-9998-8a3bea5e05d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:15.684191 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4d65c-a860-4d10-9998-8a3bea5e05d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:15.684486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4d65c-a860-4d10-9998-8a3bea5e05d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:15.684770 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4d65c-a860-4d10-9998-8a3bea5e05d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:15.702756 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c4d65c-a860-4d10-9998-8a3bea5e05d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:15.702756 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c4d65c-a860-4d10-9998-8a3bea5e05d9 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:15.707312 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1152/2304] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:36:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:36:16.300373 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4878a-77e4-42ae-a1f0-a8911b989fa6 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:16.305804 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-57b4878a-77e4-42ae-a1f0-a8911b989fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:16.316776 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b4878a-77e4-42ae-a1f0-a8911b989fa6 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:16.324590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf2249d-a341-4872-a17a-4615e1a12791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:16.325523 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2249d-a341-4872-a17a-4615e1a12791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:16.325746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2249d-a341-4872-a17a-4615e1a12791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:16.326020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf2249d-a341-4872-a17a-4615e1a12791 admin admin] PATH_INFO: `/v3/projects/6295a17c5bf646fca12c5e55b0dd7bee` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:16.327220 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf2249d-a341-4872-a17a-4615e1a12791 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6295a17c5bf646fca12c5e55b0dd7bee)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:16.329174 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf2249d-a341-4872-a17a-4615e1a12791 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:16.466255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-4cf2249d-a341-4872-a17a-4615e1a12791 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6295a17c5bf646fca12c5e55b0dd7bee', 'request_id': 'req-4cf2249d-a341-4872-a17a-4615e1a12791'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:16.472326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-4cf2249d-a341-4872-a17a-4615e1a12791 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6295a17c5bf646fca12c5e55b0dd7bee', 'request_id': 'req-4cf2249d-a341-4872-a17a-4615e1a12791'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:16.477948 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-4cf2249d-a341-4872-a17a-4615e1a12791 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6295a17c5bf646fca12c5e55b0dd7bee', 'request_id': 'req-4cf2249d-a341-4872-a17a-4615e1a12791'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:16.550284 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-4cf2249d-a341-4872-a17a-4615e1a12791 admin admin] The token cache is being invalidate because project 6295a17c5bf646fca12c5e55b0dd7bee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:36:16.550514 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-4cf2249d-a341-4872-a17a-4615e1a12791 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:16.551138 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-4cf2249d-a341-4872-a17a-4615e1a12791 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:16.552756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1153/2305] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:36:16 2022] DELETE /identity/v3/projects/6295a17c5bf646fca12c5e55b0dd7bee => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:36:16.557972 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31e3ac-ced7-42bb-b217-c362a7edb2c9 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:16.558445 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31e3ac-ced7-42bb-b217-c362a7edb2c9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:16.558614 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31e3ac-ced7-42bb-b217-c362a7edb2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:16.577157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-3e31e3ac-ced7-42bb-b217-c362a7edb2c9 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:36:16.578380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3e31e3ac-ced7-42bb-b217-c362a7edb2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:16.646843 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1153/2306] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:36:16.654968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96abbf4b-efbf-44d8-b6c3-f5516d36cd04 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:16.655567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96abbf4b-efbf-44d8-b6c3-f5516d36cd04 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:16.656012 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96abbf4b-efbf-44d8-b6c3-f5516d36cd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:16.675939 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-96abbf4b-efbf-44d8-b6c3-f5516d36cd04 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:36:16.677169 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-96abbf4b-efbf-44d8-b6c3-f5516d36cd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:16.685414 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1154/2307] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:36:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:36:16.689682 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31e3ac-ced7-42bb-b217-c362a7edb2c9 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:16.700137 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31e3ac-ced7-42bb-b217-c362a7edb2c9 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:16.704725 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d433f-71f9-415a-bd3d-ca5c78b7a7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:16.706908 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d433f-71f9-415a-bd3d-ca5c78b7a7de admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:16.706908 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d433f-71f9-415a-bd3d-ca5c78b7a7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:16.706908 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0d433f-71f9-415a-bd3d-ca5c78b7a7de admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:16.706908 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d433f-71f9-415a-bd3d-ca5c78b7a7de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:36:16.706908 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d433f-71f9-415a-bd3d-ca5c78b7a7de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:16.708651 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0d433f-71f9-415a-bd3d-ca5c78b7a7de admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:16.722458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1154/2308] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:36:16.727257 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96abbf4b-efbf-44d8-b6c3-f5516d36cd04 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:16.737777 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96abbf4b-efbf-44d8-b6c3-f5516d36cd04 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:16.742576 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221ab69-3de8-4852-807e-2529b15199a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:16.743253 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221ab69-3de8-4852-807e-2529b15199a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:16.743628 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221ab69-3de8-4852-807e-2529b15199a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:16.744578 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221ab69-3de8-4852-807e-2529b15199a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:16.745285 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221ab69-3de8-4852-807e-2529b15199a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:16.747204 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221ab69-3de8-4852-807e-2529b15199a0 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:16.760993 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:36:16.861149 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1155/2309] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:36:16 2022] POST /identity/v3/projects => generated 391 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:36:16.865198 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d433f-71f9-415a-bd3d-ca5c78b7a7de admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:16.876755 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0d433f-71f9-415a-bd3d-ca5c78b7a7de admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:16.955659 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:36:17.002906 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123510f0-cb14-4274-a6fa-f7f5405ef69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:17.003517 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123510f0-cb14-4274-a6fa-f7f5405ef69e admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:17.003803 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123510f0-cb14-4274-a6fa-f7f5405ef69e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:17.004078 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123510f0-cb14-4274-a6fa-f7f5405ef69e admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:17.004836 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123510f0-cb14-4274-a6fa-f7f5405ef69e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:17.007570 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123510f0-cb14-4274-a6fa-f7f5405ef69e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:17.145997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1155/2310] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:36:16 2022] POST /identity/v3/users => generated 296 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:36:17.147322 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221ab69-3de8-4852-807e-2529b15199a0 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:17.158460 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221ab69-3de8-4852-807e-2529b15199a0 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:17.163128 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407effcb-10b6-4d5c-bdd6-a3485cf12770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:17.163830 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407effcb-10b6-4d5c-bdd6-a3485cf12770 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:17.164116 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407effcb-10b6-4d5c-bdd6-a3485cf12770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:17.164394 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407effcb-10b6-4d5c-bdd6-a3485cf12770 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:17.165123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407effcb-10b6-4d5c-bdd6-a3485cf12770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:17.166846 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407effcb-10b6-4d5c-bdd6-a3485cf12770 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:17.177452 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1156/2311] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:36:17.183426 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123510f0-cb14-4274-a6fa-f7f5405ef69e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:17.194702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123510f0-cb14-4274-a6fa-f7f5405ef69e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:17.200780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1bc503-9460-4657-8886-f01210a83215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:17.201489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1bc503-9460-4657-8886-f01210a83215 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:17.201776 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1bc503-9460-4657-8886-f01210a83215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:17.202050 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1bc503-9460-4657-8886-f01210a83215 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:17.202793 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1bc503-9460-4657-8886-f01210a83215 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:17.204426 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1bc503-9460-4657-8886-f01210a83215 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:17.216033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1156/2312] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:36:17.220249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407effcb-10b6-4d5c-bdd6-a3485cf12770 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:17.232019 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407effcb-10b6-4d5c-bdd6-a3485cf12770 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:17.239447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee31a5-61f8-4101-ab10-4c42aa7a3efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:17.244309 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee31a5-61f8-4101-ab10-4c42aa7a3efb admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:17.244309 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee31a5-61f8-4101-ab10-4c42aa7a3efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:17.244309 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee31a5-61f8-4101-ab10-4c42aa7a3efb admin admin] PATH_INFO: `/v3/projects/04a4b6f02e1d462c9e83c22a9df3073f/users/bf63ee2d2c5343b5af92e321041103af/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:17.257202 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee31a5-61f8-4101-ab10-4c42aa7a3efb admin admin] RBAC: Authorizing `identity:create_grant(project_id=04a4b6f02e1d462c9e83c22a9df3073f, user_id=bf63ee2d2c5343b5af92e321041103af, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:17.260609 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ee31a5-61f8-4101-ab10-4c42aa7a3efb admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:17.426968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1157/2313] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:36:17 2022] PUT /identity/v3/projects/04a4b6f02e1d462c9e83c22a9df3073f/users/bf63ee2d2c5343b5af92e321041103af/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:36:17.438761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a6aa3-0139-4534-b857-31ca173fa3ce None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:17.439370 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a6aa3-0139-4534-b857-31ca173fa3ce None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:17.442014 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a6aa3-0139-4534-b857-31ca173fa3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:17.465541 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7f8a6aa3-0139-4534-b857-31ca173fa3ce None None] MFA Rules not processed for user `bf63ee2d2c5343b5af92e321041103af`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:36:17.466753 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7f8a6aa3-0139-4534-b857-31ca173fa3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:17.533648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1157/2314] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:36:17 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:36:17.545135 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee31a5-61f8-4101-ab10-4c42aa7a3efb admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:17.546975 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-98ee31a5-61f8-4101-ab10-4c42aa7a3efb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:17.604315 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee31a5-61f8-4101-ab10-4c42aa7a3efb admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:17.608017 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf066e-3c5f-46c4-a4a3-ba4611f7dc05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:17.608670 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf066e-3c5f-46c4-a4a3-ba4611f7dc05 service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:17.608961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf066e-3c5f-46c4-a4a3-ba4611f7dc05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:17.609241 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf066e-3c5f-46c4-a4a3-ba4611f7dc05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:17.617476 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf066e-3c5f-46c4-a4a3-ba4611f7dc05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:17.619647 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf066e-3c5f-46c4-a4a3-ba4611f7dc05 service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:17.628608 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1158/2315] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:36:17 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:36:20.456575 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218af1c-ac1a-4c0f-8fba-6f597910656f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:20.456575 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218af1c-ac1a-4c0f-8fba-6f597910656f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:20.456575 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5218af1c-ac1a-4c0f-8fba-6f597910656f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:20.477392 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5218af1c-ac1a-4c0f-8fba-6f597910656f None None] MFA Rules not processed for user `66ad8defb9b6453c96f1a8a7343695ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:36:20.479163 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5218af1c-ac1a-4c0f-8fba-6f597910656f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:20.551423 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1158/2316] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jun 30 10:36:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:36:20.564953 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf066e-3c5f-46c4-a4a3-ba4611f7dc05 service cinder] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:20.566472 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-b5bf066e-3c5f-46c4-a4a3-ba4611f7dc05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:20.641397 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf066e-3c5f-46c4-a4a3-ba4611f7dc05 service cinder] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:20.646411 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f820a15-a2c1-4dc8-b395-48155b6328cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07de926cb8594b098f79111dab8a9cc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:20.647061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f820a15-a2c1-4dc8-b395-48155b6328cb service swift] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:20.647411 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f820a15-a2c1-4dc8-b395-48155b6328cb service swift] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:20.647692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f820a15-a2c1-4dc8-b395-48155b6328cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:20.657151 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f820a15-a2c1-4dc8-b395-48155b6328cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:20.658760 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f820a15-a2c1-4dc8-b395-48155b6328cb service swift] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:20.668142 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1159/2317] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jun 30 10:36:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:36:30.540197 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=77,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:36:30.665252 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:INSERT=2,keystone:SELECT=75 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:36:38.580715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218af1c-ac1a-4c0f-8fba-6f597910656f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:38.580715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5218af1c-ac1a-4c0f-8fba-6f597910656f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:38.708207 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5218af1c-ac1a-4c0f-8fba-6f597910656f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:38.714301 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334d6d88-449a-412d-89f8-23635754920d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46c99f43a9344d7d92632990e45bb9cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:38.715080 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d6d88-449a-412d-89f8-23635754920d service placement] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:38.715403 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d6d88-449a-412d-89f8-23635754920d service placement] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:38.716073 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334d6d88-449a-412d-89f8-23635754920d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:38.718115 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-334d6d88-449a-412d-89f8-23635754920d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:38.731012 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d6d88-449a-412d-89f8-23635754920d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:38.732727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334d6d88-449a-412d-89f8-23635754920d service placement] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:38.743216 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1159/2318] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:36:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:36:46.620556 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f820a15-a2c1-4dc8-b395-48155b6328cb service swift] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:46.678726 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f820a15-a2c1-4dc8-b395-48155b6328cb service swift] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:46.683613 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47af46-b2d1-4eba-afad-ac7f65aef737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:46.684489 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47af46-b2d1-4eba-afad-ac7f65aef737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:46.684776 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47af46-b2d1-4eba-afad-ac7f65aef737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:46.685055 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b47af46-b2d1-4eba-afad-ac7f65aef737 admin admin] PATH_INFO: `/v3/users/bf63ee2d2c5343b5af92e321041103af` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:46.686246 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b47af46-b2d1-4eba-afad-ac7f65aef737 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf63ee2d2c5343b5af92e321041103af)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:46.688216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b47af46-b2d1-4eba-afad-ac7f65aef737 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:46.865400 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-8b47af46-b2d1-4eba-afad-ac7f65aef737 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf63ee2d2c5343b5af92e321041103af'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:46.949614 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-8b47af46-b2d1-4eba-afad-ac7f65aef737 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf63ee2d2c5343b5af92e321041103af'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:46.961896 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-8b47af46-b2d1-4eba-afad-ac7f65aef737 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf63ee2d2c5343b5af92e321041103af'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:46.962545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-8b47af46-b2d1-4eba-afad-ac7f65aef737 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf63ee2d2c5343b5af92e321041103af'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:46.963181 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-8b47af46-b2d1-4eba-afad-ac7f65aef737 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf63ee2d2c5343b5af92e321041103af'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:46.975569 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1160/2319] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:36:46 2022] DELETE /identity/v3/users/bf63ee2d2c5343b5af92e321041103af => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:36:46.981753 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895f239-db77-46d7-86c0-fd4b75fde7e2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:46.981753 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895f239-db77-46d7-86c0-fd4b75fde7e2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:46.981753 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9895f239-db77-46d7-86c0-fd4b75fde7e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:46.999413 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9895f239-db77-46d7-86c0-fd4b75fde7e2 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:36:47.000775 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9895f239-db77-46d7-86c0-fd4b75fde7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:47.060669 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1160/2320] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:36:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:36:47.070500 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47af46-b2d1-4eba-afad-ac7f65aef737 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:47.072265 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-8b47af46-b2d1-4eba-afad-ac7f65aef737 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:47.129998 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b47af46-b2d1-4eba-afad-ac7f65aef737 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:47.133670 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb54383-a2b9-411e-b11b-c6850ecf2d55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:47.134314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb54383-a2b9-411e-b11b-c6850ecf2d55 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:47.134599 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb54383-a2b9-411e-b11b-c6850ecf2d55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:47.135519 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb54383-a2b9-411e-b11b-c6850ecf2d55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:47.145249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb54383-a2b9-411e-b11b-c6850ecf2d55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:47.146679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb54383-a2b9-411e-b11b-c6850ecf2d55 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:47.157348 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1161/2321] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:36:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:36:47.612686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895f239-db77-46d7-86c0-fd4b75fde7e2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:47.614365 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9895f239-db77-46d7-86c0-fd4b75fde7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:47.647342 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9895f239-db77-46d7-86c0-fd4b75fde7e2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:47.656026 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584604a-d665-4dd4-8ec8-d458b6d321eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:47.657671 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584604a-d665-4dd4-8ec8-d458b6d321eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:47.657671 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584604a-d665-4dd4-8ec8-d458b6d321eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:47.658483 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584604a-d665-4dd4-8ec8-d458b6d321eb admin admin] PATH_INFO: `/v3/projects/04a4b6f02e1d462c9e83c22a9df3073f` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:47.659516 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584604a-d665-4dd4-8ec8-d458b6d321eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=04a4b6f02e1d462c9e83c22a9df3073f)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:47.662064 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584604a-d665-4dd4-8ec8-d458b6d321eb admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:47.844806 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f584604a-d665-4dd4-8ec8-d458b6d321eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04a4b6f02e1d462c9e83c22a9df3073f', 'request_id': 'req-f584604a-d665-4dd4-8ec8-d458b6d321eb'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:47.949001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f584604a-d665-4dd4-8ec8-d458b6d321eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04a4b6f02e1d462c9e83c22a9df3073f', 'request_id': 'req-f584604a-d665-4dd4-8ec8-d458b6d321eb'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:47.955121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f584604a-d665-4dd4-8ec8-d458b6d321eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04a4b6f02e1d462c9e83c22a9df3073f', 'request_id': 'req-f584604a-d665-4dd4-8ec8-d458b6d321eb'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:47.956390 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:36:47.968503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f584604a-d665-4dd4-8ec8-d458b6d321eb admin admin] The token cache is being invalidate because project 04a4b6f02e1d462c9e83c22a9df3073f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:36:47.968789 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f584604a-d665-4dd4-8ec8-d458b6d321eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:47.969439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f584604a-d665-4dd4-8ec8-d458b6d321eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:47.972009 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1161/2322] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:36:47 2022] DELETE /identity/v3/projects/04a4b6f02e1d462c9e83c22a9df3073f => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:36:47.976432 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79602677-142e-4f93-a3ee-353b79c74886 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:47.986328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79602677-142e-4f93-a3ee-353b79c74886 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:47.986328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79602677-142e-4f93-a3ee-353b79c74886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:47.995365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-79602677-142e-4f93-a3ee-353b79c74886 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:36:47.996573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-79602677-142e-4f93-a3ee-353b79c74886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:48.013265 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:36:48.055996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1162/2323] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:36:48.064013 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36aff2a-aaa4-4604-846b-c9bbe6e31f10 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:48.064545 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36aff2a-aaa4-4604-846b-c9bbe6e31f10 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:48.064545 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36aff2a-aaa4-4604-846b-c9bbe6e31f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:48.083736 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f36aff2a-aaa4-4604-846b-c9bbe6e31f10 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:36:48.084940 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f36aff2a-aaa4-4604-846b-c9bbe6e31f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:48.093944 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1162/2324] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:36:48.098176 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79602677-142e-4f93-a3ee-353b79c74886 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:48.108735 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79602677-142e-4f93-a3ee-353b79c74886 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:48.113334 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33440ef-9d7c-4799-afda-ddfb198eb8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:48.113999 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33440ef-9d7c-4799-afda-ddfb198eb8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:48.114280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33440ef-9d7c-4799-afda-ddfb198eb8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:48.114582 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33440ef-9d7c-4799-afda-ddfb198eb8d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:48.115502 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33440ef-9d7c-4799-afda-ddfb198eb8d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:36:48.115821 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33440ef-9d7c-4799-afda-ddfb198eb8d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:48.118803 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33440ef-9d7c-4799-afda-ddfb198eb8d1 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:48.129115 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1163/2325] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:36:48.133596 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36aff2a-aaa4-4604-846b-c9bbe6e31f10 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:48.143569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36aff2a-aaa4-4604-846b-c9bbe6e31f10 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:48.148175 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80edb8-64f2-4d7f-b1cc-cd88efe3031c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:48.155975 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80edb8-64f2-4d7f-b1cc-cd88efe3031c admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:48.155975 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80edb8-64f2-4d7f-b1cc-cd88efe3031c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:48.155975 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80edb8-64f2-4d7f-b1cc-cd88efe3031c admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:48.155975 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae80edb8-64f2-4d7f-b1cc-cd88efe3031c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:48.155975 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae80edb8-64f2-4d7f-b1cc-cd88efe3031c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:48.288867 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1163/2326] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:36:48 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:36:48.293413 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33440ef-9d7c-4799-afda-ddfb198eb8d1 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:48.303192 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33440ef-9d7c-4799-afda-ddfb198eb8d1 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:48.362065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba664db-d40d-4c0d-8db8-8c55ac6591b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:48.362741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba664db-d40d-4c0d-8db8-8c55ac6591b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:48.363056 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba664db-d40d-4c0d-8db8-8c55ac6591b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:48.363368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba664db-d40d-4c0d-8db8-8c55ac6591b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:48.364516 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba664db-d40d-4c0d-8db8-8c55ac6591b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:48.366345 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba664db-d40d-4c0d-8db8-8c55ac6591b9 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:48.446271 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1164/2327] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:36:48 2022] POST /identity/v3/users => generated 287 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:36:48.451305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80edb8-64f2-4d7f-b1cc-cd88efe3031c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:48.461808 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80edb8-64f2-4d7f-b1cc-cd88efe3031c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:48.467238 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d838a7-91df-4244-bc48-3277ef625faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:48.468132 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d838a7-91df-4244-bc48-3277ef625faa admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:48.470095 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d838a7-91df-4244-bc48-3277ef625faa admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:48.470095 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d838a7-91df-4244-bc48-3277ef625faa admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:48.470095 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d838a7-91df-4244-bc48-3277ef625faa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:48.471362 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d838a7-91df-4244-bc48-3277ef625faa admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:48.482054 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1164/2328] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:36:48.490259 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba664db-d40d-4c0d-8db8-8c55ac6591b9 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:48.501952 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba664db-d40d-4c0d-8db8-8c55ac6591b9 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:48.507095 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af72075-b9b9-4746-b1ed-cc17bb82929e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:48.507790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af72075-b9b9-4746-b1ed-cc17bb82929e admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:48.508077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af72075-b9b9-4746-b1ed-cc17bb82929e admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:48.508392 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af72075-b9b9-4746-b1ed-cc17bb82929e admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:48.509151 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af72075-b9b9-4746-b1ed-cc17bb82929e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:48.511420 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af72075-b9b9-4746-b1ed-cc17bb82929e admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:48.521503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1165/2329] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:36:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:36:48.525665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d838a7-91df-4244-bc48-3277ef625faa admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:48.535923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d838a7-91df-4244-bc48-3277ef625faa admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:48.541286 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4be55b-843e-4430-b3dc-79e8eeb60bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:48.542095 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4be55b-843e-4430-b3dc-79e8eeb60bc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:48.542525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4be55b-843e-4430-b3dc-79e8eeb60bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:48.542808 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4be55b-843e-4430-b3dc-79e8eeb60bc3 admin admin] PATH_INFO: `/v3/projects/3a35109cec4141168620f42478a95c9c/users/1b151de186ff48b193672ec104e031a1/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:48.560842 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4be55b-843e-4430-b3dc-79e8eeb60bc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a35109cec4141168620f42478a95c9c, user_id=1b151de186ff48b193672ec104e031a1, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:48.565034 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4be55b-843e-4430-b3dc-79e8eeb60bc3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:48.664470 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1165/2330] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:36:48 2022] PUT /identity/v3/projects/3a35109cec4141168620f42478a95c9c/users/1b151de186ff48b193672ec104e031a1/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:36:48.665838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe11fe-6597-4cfd-92c8-256f9fa9be09 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:48.666758 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe11fe-6597-4cfd-92c8-256f9fa9be09 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:48.667101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe11fe-6597-4cfd-92c8-256f9fa9be09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:48.689069 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-9ffe11fe-6597-4cfd-92c8-256f9fa9be09 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:36:48.690281 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-9ffe11fe-6597-4cfd-92c8-256f9fa9be09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:48.767742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1166/2331] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:36:48.779002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4be55b-843e-4430-b3dc-79e8eeb60bc3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:48.781463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8b4be55b-843e-4430-b3dc-79e8eeb60bc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:48.845713 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4be55b-843e-4430-b3dc-79e8eeb60bc3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:48.850317 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811514ee-d247-43bb-bdfd-8465fb87e1aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:48.850317 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811514ee-d247-43bb-bdfd-8465fb87e1aa service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:48.850989 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811514ee-d247-43bb-bdfd-8465fb87e1aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:48.851378 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811514ee-d247-43bb-bdfd-8465fb87e1aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:48.864594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811514ee-d247-43bb-bdfd-8465fb87e1aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:48.866513 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811514ee-d247-43bb-bdfd-8465fb87e1aa service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:48.891446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1166/2332] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:36:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:36:50.218092 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe11fe-6597-4cfd-92c8-256f9fa9be09 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:50.219598 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-9ffe11fe-6597-4cfd-92c8-256f9fa9be09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:50.232497 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe11fe-6597-4cfd-92c8-256f9fa9be09 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:50.237087 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12b627-4434-47fc-bab6-d967ff97b97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:50.237975 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12b627-4434-47fc-bab6-d967ff97b97b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:50.238257 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12b627-4434-47fc-bab6-d967ff97b97b admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:50.238640 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12b627-4434-47fc-bab6-d967ff97b97b admin admin] PATH_INFO: `/v3/users/6822ff27450e4218b2811f2cf2d41999` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:50.239828 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12b627-4434-47fc-bab6-d967ff97b97b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6822ff27450e4218b2811f2cf2d41999)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:50.241767 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a12b627-4434-47fc-bab6-d967ff97b97b admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:50.268648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:36:50.434834 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-7a12b627-4434-47fc-bab6-d967ff97b97b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6822ff27450e4218b2811f2cf2d41999'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:50.530175 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-7a12b627-4434-47fc-bab6-d967ff97b97b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6822ff27450e4218b2811f2cf2d41999'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:50.544236 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-7a12b627-4434-47fc-bab6-d967ff97b97b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6822ff27450e4218b2811f2cf2d41999'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:50.545036 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-7a12b627-4434-47fc-bab6-d967ff97b97b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6822ff27450e4218b2811f2cf2d41999'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:50.545654 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-7a12b627-4434-47fc-bab6-d967ff97b97b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6822ff27450e4218b2811f2cf2d41999'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:50.559606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1167/2333] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:36:50 2022] DELETE /identity/v3/users/6822ff27450e4218b2811f2cf2d41999 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:36:50.564138 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db322c-2ff2-4d89-a0f0-884be383abd8 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:50.564627 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db322c-2ff2-4d89-a0f0-884be383abd8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:50.564720 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4db322c-2ff2-4d89-a0f0-884be383abd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:50.582456 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a4db322c-2ff2-4d89-a0f0-884be383abd8 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:36:50.583674 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a4db322c-2ff2-4d89-a0f0-884be383abd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:50.648070 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1167/2334] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:36:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:36:50.657787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12b627-4434-47fc-bab6-d967ff97b97b admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:50.661494 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-7a12b627-4434-47fc-bab6-d967ff97b97b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:50.718700 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a12b627-4434-47fc-bab6-d967ff97b97b admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:50.724992 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd20668-53c7-4e3b-b002-4e2b55070d43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:50.725670 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd20668-53c7-4e3b-b002-4e2b55070d43 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:50.725958 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd20668-53c7-4e3b-b002-4e2b55070d43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:50.726241 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd20668-53c7-4e3b-b002-4e2b55070d43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:50.738181 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd20668-53c7-4e3b-b002-4e2b55070d43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:50.739674 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd20668-53c7-4e3b-b002-4e2b55070d43 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:50.751561 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1168/2335] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:36:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:36:51.257632 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db322c-2ff2-4d89-a0f0-884be383abd8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:51.258825 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a4db322c-2ff2-4d89-a0f0-884be383abd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:51.271868 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4db322c-2ff2-4d89-a0f0-884be383abd8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:51.285025 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393431d-786f-4e71-a760-1b6677449903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:51.286162 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393431d-786f-4e71-a760-1b6677449903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:51.286450 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393431d-786f-4e71-a760-1b6677449903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:51.286906 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393431d-786f-4e71-a760-1b6677449903 admin admin] PATH_INFO: `/v3/projects/77b41d8f0e4d4eadbc5947357d2d0c44` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:51.288789 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393431d-786f-4e71-a760-1b6677449903 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77b41d8f0e4d4eadbc5947357d2d0c44)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:51.290717 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393431d-786f-4e71-a760-1b6677449903 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:51.399598 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:36:51.438789 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f393431d-786f-4e71-a760-1b6677449903 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77b41d8f0e4d4eadbc5947357d2d0c44', 'request_id': 'req-f393431d-786f-4e71-a760-1b6677449903'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:51.538731 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f393431d-786f-4e71-a760-1b6677449903 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77b41d8f0e4d4eadbc5947357d2d0c44', 'request_id': 'req-f393431d-786f-4e71-a760-1b6677449903'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:51.546862 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f393431d-786f-4e71-a760-1b6677449903 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77b41d8f0e4d4eadbc5947357d2d0c44', 'request_id': 'req-f393431d-786f-4e71-a760-1b6677449903'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:51.561138 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f393431d-786f-4e71-a760-1b6677449903 admin admin] The token cache is being invalidate because project 77b41d8f0e4d4eadbc5947357d2d0c44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:36:51.561667 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f393431d-786f-4e71-a760-1b6677449903 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:51.562505 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f393431d-786f-4e71-a760-1b6677449903 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:36:51.564865 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1168/2336] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:36:51 2022] DELETE /identity/v3/projects/77b41d8f0e4d4eadbc5947357d2d0c44 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:36:51.569543 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdbcd4-e185-4b98-aa51-2688bd18eba7 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:51.570346 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdbcd4-e185-4b98-aa51-2688bd18eba7 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:51.570664 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdbcd4-e185-4b98-aa51-2688bd18eba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:51.591901 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-44bdbcd4-e185-4b98-aa51-2688bd18eba7 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:36:51.593245 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-44bdbcd4-e185-4b98-aa51-2688bd18eba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:51.664185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1169/2337] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:36:51.672233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f52d3c-09c4-41ba-bda9-509ef9947fa1 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:51.673935 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f52d3c-09c4-41ba-bda9-509ef9947fa1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:51.673935 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f52d3c-09c4-41ba-bda9-509ef9947fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:51.700795 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f4f52d3c-09c4-41ba-bda9-509ef9947fa1 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:36:51.702493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f4f52d3c-09c4-41ba-bda9-509ef9947fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:51.715946 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1169/2338] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:36:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:36:51.721587 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bdbcd4-e185-4b98-aa51-2688bd18eba7 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:51.734999 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bdbcd4-e185-4b98-aa51-2688bd18eba7 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:51.740797 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a076f0-a891-40c8-8478-d5a4e6572ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:51.741871 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a076f0-a891-40c8-8478-d5a4e6572ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:51.742353 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a076f0-a891-40c8-8478-d5a4e6572ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:51.742817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a076f0-a891-40c8-8478-d5a4e6572ab2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:51.744503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a076f0-a891-40c8-8478-d5a4e6572ab2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:36:51.744972 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a076f0-a891-40c8-8478-d5a4e6572ab2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:51.747290 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a076f0-a891-40c8-8478-d5a4e6572ab2 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:51.765487 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1170/2339] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:36:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:36:51.769708 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f52d3c-09c4-41ba-bda9-509ef9947fa1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:51.782941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f52d3c-09c4-41ba-bda9-509ef9947fa1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:51.803527 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009940d-8c9a-447c-8a46-64797dab936b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:51.804343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009940d-8c9a-447c-8a46-64797dab936b admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:51.804699 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009940d-8c9a-447c-8a46-64797dab936b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:51.805044 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a009940d-8c9a-447c-8a46-64797dab936b admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:51.807761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a009940d-8c9a-447c-8a46-64797dab936b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:51.811931 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a009940d-8c9a-447c-8a46-64797dab936b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:51.915068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1170/2340] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:36:51 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:36:51.919822 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a076f0-a891-40c8-8478-d5a4e6572ab2 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:51.944131 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a076f0-a891-40c8-8478-d5a4e6572ab2 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:52.004385 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:36:52.046163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d56a0-0a1e-4186-9c75-3e7838499709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:52.046163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d56a0-0a1e-4186-9c75-3e7838499709 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:52.046163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d56a0-0a1e-4186-9c75-3e7838499709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:52.046163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d56a0-0a1e-4186-9c75-3e7838499709 admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:52.046815 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d56a0-0a1e-4186-9c75-3e7838499709 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:52.048055 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d56a0-0a1e-4186-9c75-3e7838499709 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:52.164403 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009940d-8c9a-447c-8a46-64797dab936b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:52.167764 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1171/2341] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:36:51 2022] POST /identity/v3/users => generated 287 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:36:52.182622 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a009940d-8c9a-447c-8a46-64797dab936b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:52.189158 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf0132-7bc9-425f-9159-0f58df6f641b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:52.190077 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf0132-7bc9-425f-9159-0f58df6f641b admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:52.190423 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf0132-7bc9-425f-9159-0f58df6f641b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:52.190762 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bf0132-7bc9-425f-9159-0f58df6f641b admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:52.191748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf0132-7bc9-425f-9159-0f58df6f641b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:52.193641 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bf0132-7bc9-425f-9159-0f58df6f641b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:52.221561 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1171/2342] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:36:52.225588 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d56a0-0a1e-4186-9c75-3e7838499709 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:52.241545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d56a0-0a1e-4186-9c75-3e7838499709 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:52.248820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3349754-1dd8-4b43-8cec-9b6864a44bf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:52.249567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3349754-1dd8-4b43-8cec-9b6864a44bf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:52.249858 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3349754-1dd8-4b43-8cec-9b6864a44bf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:52.250136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3349754-1dd8-4b43-8cec-9b6864a44bf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:52.251118 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3349754-1dd8-4b43-8cec-9b6864a44bf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:52.253110 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3349754-1dd8-4b43-8cec-9b6864a44bf3 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:52.274778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf0132-7bc9-425f-9159-0f58df6f641b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:52.278494 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1172/2343] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:36:52.286956 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bf0132-7bc9-425f-9159-0f58df6f641b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:52.293985 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153d54e-81dd-4322-ac5d-107503563805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:52.294940 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153d54e-81dd-4322-ac5d-107503563805 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:52.295228 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153d54e-81dd-4322-ac5d-107503563805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:52.295533 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e153d54e-81dd-4322-ac5d-107503563805 admin admin] PATH_INFO: `/v3/projects/e1464cdd99e64bbeab3d8f44b6b6e0ab/users/1a5c0e063318493f918184ec2f18ef1b/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:52.313255 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e153d54e-81dd-4322-ac5d-107503563805 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1464cdd99e64bbeab3d8f44b6b6e0ab, user_id=1a5c0e063318493f918184ec2f18ef1b, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:52.316692 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e153d54e-81dd-4322-ac5d-107503563805 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:52.393253 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1172/2344] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:36:52 2022] PUT /identity/v3/projects/e1464cdd99e64bbeab3d8f44b6b6e0ab/users/1a5c0e063318493f918184ec2f18ef1b/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:36:52.405108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f8604-dbea-40da-88cc-1401a871e4cf None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:52.405658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f8604-dbea-40da-88cc-1401a871e4cf None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:52.405942 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663f8604-dbea-40da-88cc-1401a871e4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:52.444798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-663f8604-dbea-40da-88cc-1401a871e4cf None None] MFA Rules not processed for user `1a5c0e063318493f918184ec2f18ef1b`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:36:52.446206 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-663f8604-dbea-40da-88cc-1401a871e4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:52.536196 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1173/2345] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:36:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:36:52.555984 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153d54e-81dd-4322-ac5d-107503563805 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:52.558897 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e153d54e-81dd-4322-ac5d-107503563805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:52.641785 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e153d54e-81dd-4322-ac5d-107503563805 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:52.645609 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea8be3-b49f-4c08-916b-13ce67fbfef3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:52.646276 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea8be3-b49f-4c08-916b-13ce67fbfef3 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:52.646561 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea8be3-b49f-4c08-916b-13ce67fbfef3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:52.646836 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acea8be3-b49f-4c08-916b-13ce67fbfef3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:52.668241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea8be3-b49f-4c08-916b-13ce67fbfef3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:52.678669 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acea8be3-b49f-4c08-916b-13ce67fbfef3 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:52.693594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1173/2346] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:36:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:36:57.093933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304241a6-6b06-475b-8f9b-30a35b01b002 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:57.094546 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304241a6-6b06-475b-8f9b-30a35b01b002 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:57.094546 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304241a6-6b06-475b-8f9b-30a35b01b002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:57.122631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-304241a6-6b06-475b-8f9b-30a35b01b002 None None] MFA Rules not processed for user `1b151de186ff48b193672ec104e031a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:36:57.123918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-304241a6-6b06-475b-8f9b-30a35b01b002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:57.196088 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1174/2347] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:36:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:36:57.200460 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea8be3-b49f-4c08-916b-13ce67fbfef3 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:57.201982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-acea8be3-b49f-4c08-916b-13ce67fbfef3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:36:57.285622 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:36:57.291522 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acea8be3-b49f-4c08-916b-13ce67fbfef3 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:57.300143 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b361170-6461-485b-9113-39f26b00981d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:57.300828 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b361170-6461-485b-9113-39f26b00981d admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:57.301128 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b361170-6461-485b-9113-39f26b00981d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:57.301392 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b361170-6461-485b-9113-39f26b00981d admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:57.302106 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b361170-6461-485b-9113-39f26b00981d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:57.304168 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b361170-6461-485b-9113-39f26b00981d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:57.412665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1174/2348] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:36:57 2022] POST /identity/v3/projects => generated 373 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:36:57.413768 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304241a6-6b06-475b-8f9b-30a35b01b002 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:57.425038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304241a6-6b06-475b-8f9b-30a35b01b002 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:57.453497 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:36:57.486213 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0556b5-35c4-499b-9b86-2e390d6f65e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:57.486893 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0556b5-35c4-499b-9b86-2e390d6f65e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:57.487177 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0556b5-35c4-499b-9b86-2e390d6f65e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:57.487503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0556b5-35c4-499b-9b86-2e390d6f65e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:57.488209 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0556b5-35c4-499b-9b86-2e390d6f65e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:57.490247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0556b5-35c4-499b-9b86-2e390d6f65e6 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:57.606672 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1175/2349] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:36:57 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:36:57.610892 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b361170-6461-485b-9113-39f26b00981d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:57.623538 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b361170-6461-485b-9113-39f26b00981d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:57.628153 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5120ee9-2d08-45f6-9b62-bb9849e9b99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:57.628824 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5120ee9-2d08-45f6-9b62-bb9849e9b99c admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:57.629113 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5120ee9-2d08-45f6-9b62-bb9849e9b99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:57.629394 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5120ee9-2d08-45f6-9b62-bb9849e9b99c admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:57.630130 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5120ee9-2d08-45f6-9b62-bb9849e9b99c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:57.631835 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5120ee9-2d08-45f6-9b62-bb9849e9b99c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:57.644112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1175/2350] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:36:57.648689 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0556b5-35c4-499b-9b86-2e390d6f65e6 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:57.660513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0556b5-35c4-499b-9b86-2e390d6f65e6 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:57.665620 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572a5aa-924a-4f8c-8876-6ad779b98b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:57.670733 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572a5aa-924a-4f8c-8876-6ad779b98b68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:57.670733 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572a5aa-924a-4f8c-8876-6ad779b98b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:57.670733 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572a5aa-924a-4f8c-8876-6ad779b98b68 admin admin] PATH_INFO: `/v3/projects/3a5f173a69c742e4aba26537e5b25d52/users/fd5adf3f3cda419c8cc1c82d2c20f361/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:57.683594 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572a5aa-924a-4f8c-8876-6ad779b98b68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a5f173a69c742e4aba26537e5b25d52, user_id=fd5adf3f3cda419c8cc1c82d2c20f361, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:57.686291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572a5aa-924a-4f8c-8876-6ad779b98b68 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:57.780961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1176/2351] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:36:57 2022] PUT /identity/v3/projects/3a5f173a69c742e4aba26537e5b25d52/users/fd5adf3f3cda419c8cc1c82d2c20f361/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:36:57.786431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5120ee9-2d08-45f6-9b62-bb9849e9b99c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:57.800982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5120ee9-2d08-45f6-9b62-bb9849e9b99c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:57.919608 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e8aad-aa1d-4cd8-a0a5-4a4b6d2a4bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:57.920487 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e8aad-aa1d-4cd8-a0a5-4a4b6d2a4bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:57.920780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e8aad-aa1d-4cd8-a0a5-4a4b6d2a4bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:57.921219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e8aad-aa1d-4cd8-a0a5-4a4b6d2a4bf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:57.922059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e8aad-aa1d-4cd8-a0a5-4a4b6d2a4bf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:57.923954 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e8aad-aa1d-4cd8-a0a5-4a4b6d2a4bf4 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:57.935621 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1176/2352] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:36:57.939377 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572a5aa-924a-4f8c-8876-6ad779b98b68 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:36:57.951783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572a5aa-924a-4f8c-8876-6ad779b98b68 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:36:57.956667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d481f45a-f657-41d2-b912-a3c6e63cf98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:36:57.957552 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481f45a-f657-41d2-b912-a3c6e63cf98a admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:36:57.957862 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481f45a-f657-41d2-b912-a3c6e63cf98a admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:36:57.958111 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481f45a-f657-41d2-b912-a3c6e63cf98a admin admin] PATH_INFO: `/v3/projects/3a5f173a69c742e4aba26537e5b25d52/users/fd5adf3f3cda419c8cc1c82d2c20f361/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:36:57.960323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d481f45a-f657-41d2-b912-a3c6e63cf98a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a5f173a69c742e4aba26537e5b25d52, user_id=fd5adf3f3cda419c8cc1c82d2c20f361, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:36:57.963694 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d481f45a-f657-41d2-b912-a3c6e63cf98a admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:36:57.985871 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1177/2353] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:36:57 2022] PUT /identity/v3/projects/3a5f173a69c742e4aba26537e5b25d52/users/fd5adf3f3cda419c8cc1c82d2c20f361/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:37:04.324872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e9d65-232f-46ce-b7af-7ecfafd71935 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:37:04.326219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e9d65-232f-46ce-b7af-7ecfafd71935 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:37:04.326564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e9d65-232f-46ce-b7af-7ecfafd71935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:37:04.386770 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ba7e9d65-232f-46ce-b7af-7ecfafd71935 None None] MFA Rules not processed for user `fd5adf3f3cda419c8cc1c82d2c20f361`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:37:04.388120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ba7e9d65-232f-46ce-b7af-7ecfafd71935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:37:04.493369 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1177/2354] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:37:04 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:37:04.511897 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d481f45a-f657-41d2-b912-a3c6e63cf98a admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:37:04.514521 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d481f45a-f657-41d2-b912-a3c6e63cf98a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:37:04.646496 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d481f45a-f657-41d2-b912-a3c6e63cf98a admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:37:04.654907 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939f54c9-d17f-49b3-88c2-a3454964eff1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:37:04.655673 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f54c9-d17f-49b3-88c2-a3454964eff1 service nova] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:37:04.655990 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f54c9-d17f-49b3-88c2-a3454964eff1 service nova] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:37:04.656271 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939f54c9-d17f-49b3-88c2-a3454964eff1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:37:04.749450 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f54c9-d17f-49b3-88c2-a3454964eff1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:37:04.751108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939f54c9-d17f-49b3-88c2-a3454964eff1 service nova] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:37:04.755586 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:37:04.792447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1178/2355] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:37:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:37:05.771835 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e9d65-232f-46ce-b7af-7ecfafd71935 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:37:05.773912 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ba7e9d65-232f-46ce-b7af-7ecfafd71935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:37:05.835881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e9d65-232f-46ce-b7af-7ecfafd71935 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:37:05.840372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750de492-194a-483a-9abb-4669e6766182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:37:05.841062 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750de492-194a-483a-9abb-4669e6766182 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:37:05.841352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750de492-194a-483a-9abb-4669e6766182 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:37:05.841632 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750de492-194a-483a-9abb-4669e6766182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:37:05.843504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-750de492-194a-483a-9abb-4669e6766182 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:37:05.850081 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:37:05.916022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750de492-194a-483a-9abb-4669e6766182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:37:05.917080 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750de492-194a-483a-9abb-4669e6766182 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:37:05.928186 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1178/2356] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:37:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:37:08.351481 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45c06f-ca51-40f7-94c5-13c0d4d4b1ec None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:37:08.352705 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45c06f-ca51-40f7-94c5-13c0d4d4b1ec None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:37:08.353699 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c45c06f-ca51-40f7-94c5-13c0d4d4b1ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:37:08.373764 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-4c45c06f-ca51-40f7-94c5-13c0d4d4b1ec None None] MFA Rules not processed for user `4d368bff9f91483b81e6a192f7f90ff5`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:37:08.375432 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-4c45c06f-ca51-40f7-94c5-13c0d4d4b1ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:37:08.537690 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1179/2357] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:37:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:37:09.682807 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc178e48-1781-47f2-bd03-756c962b863b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:37:09.688291 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc178e48-1781-47f2-bd03-756c962b863b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:37:09.688291 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc178e48-1781-47f2-bd03-756c962b863b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:37:09.725934 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-fc178e48-1781-47f2-bd03-756c962b863b None None] MFA Rules not processed for user `1b151de186ff48b193672ec104e031a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:37:09.727621 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fc178e48-1781-47f2-bd03-756c962b863b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:37:09.761595 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1179/2358] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:37:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:37:12.745288 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c45c06f-ca51-40f7-94c5-13c0d4d4b1ec None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:37:12.830630 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c45c06f-ca51-40f7-94c5-13c0d4d4b1ec None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:37:12.839955 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa4ede-6d4e-4cc7-b115-b9d126d58568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:37:12.840828 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa4ede-6d4e-4cc7-b115-b9d126d58568 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:37:12.841120 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa4ede-6d4e-4cc7-b115-b9d126d58568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:37:12.841403 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa4ede-6d4e-4cc7-b115-b9d126d58568 admin admin] PATH_INFO: `/v3/users/1a5c0e063318493f918184ec2f18ef1b` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:37:12.842663 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa4ede-6d4e-4cc7-b115-b9d126d58568 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a5c0e063318493f918184ec2f18ef1b)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:37:12.844647 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa4ede-6d4e-4cc7-b115-b9d126d58568 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:37:13.118109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9baa4ede-6d4e-4cc7-b115-b9d126d58568 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a5c0e063318493f918184ec2f18ef1b'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:37:13.196013 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9baa4ede-6d4e-4cc7-b115-b9d126d58568 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a5c0e063318493f918184ec2f18ef1b'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:37:13.216502 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9baa4ede-6d4e-4cc7-b115-b9d126d58568 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a5c0e063318493f918184ec2f18ef1b'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:37:13.217348 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9baa4ede-6d4e-4cc7-b115-b9d126d58568 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a5c0e063318493f918184ec2f18ef1b'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:37:13.217975 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9baa4ede-6d4e-4cc7-b115-b9d126d58568 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a5c0e063318493f918184ec2f18ef1b'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:37:13.247340 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1180/2359] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:37:12 2022] DELETE /identity/v3/users/1a5c0e063318493f918184ec2f18ef1b => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:37:13.249775 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a6902-c636-471c-b75d-956a26d4af5a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:37:13.253878 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a6902-c636-471c-b75d-956a26d4af5a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:37:13.253878 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a6902-c636-471c-b75d-956a26d4af5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:37:13.272359 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5c4a6902-c636-471c-b75d-956a26d4af5a None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:37:13.273671 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5c4a6902-c636-471c-b75d-956a26d4af5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:37:13.353530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1180/2360] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:37:13.365776 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa4ede-6d4e-4cc7-b115-b9d126d58568 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:37:13.367363 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-9baa4ede-6d4e-4cc7-b115-b9d126d58568 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:37:13.516327 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:37:13.596022 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa4ede-6d4e-4cc7-b115-b9d126d58568 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:37:13.612377 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41431a40-78dd-4d97-9b88-eccb07277379 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:37:13.613064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41431a40-78dd-4d97-9b88-eccb07277379 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:37:13.613342 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41431a40-78dd-4d97-9b88-eccb07277379 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:37:13.613617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41431a40-78dd-4d97-9b88-eccb07277379 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:37:13.627210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41431a40-78dd-4d97-9b88-eccb07277379 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:37:13.628893 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41431a40-78dd-4d97-9b88-eccb07277379 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:37:13.661134 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1181/2361] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:37:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:37:14.729068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a6902-c636-471c-b75d-956a26d4af5a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:37:14.730580 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5c4a6902-c636-471c-b75d-956a26d4af5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:37:14.743087 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4a6902-c636-471c-b75d-956a26d4af5a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:37:14.748961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb781b8-86cd-4dd2-a67f-12be8af2802b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:37:14.749840 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb781b8-86cd-4dd2-a67f-12be8af2802b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:37:14.750122 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb781b8-86cd-4dd2-a67f-12be8af2802b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:37:14.750404 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb781b8-86cd-4dd2-a67f-12be8af2802b admin admin] PATH_INFO: `/v3/projects/e1464cdd99e64bbeab3d8f44b6b6e0ab` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:37:14.752372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb781b8-86cd-4dd2-a67f-12be8af2802b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1464cdd99e64bbeab3d8f44b6b6e0ab)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:37:14.754933 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb781b8-86cd-4dd2-a67f-12be8af2802b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:37:14.895651 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5bb781b8-86cd-4dd2-a67f-12be8af2802b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1464cdd99e64bbeab3d8f44b6b6e0ab', 'request_id': 'req-5bb781b8-86cd-4dd2-a67f-12be8af2802b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:37:14.984470 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5bb781b8-86cd-4dd2-a67f-12be8af2802b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1464cdd99e64bbeab3d8f44b6b6e0ab', 'request_id': 'req-5bb781b8-86cd-4dd2-a67f-12be8af2802b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:37:14.990622 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5bb781b8-86cd-4dd2-a67f-12be8af2802b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1464cdd99e64bbeab3d8f44b6b6e0ab', 'request_id': 'req-5bb781b8-86cd-4dd2-a67f-12be8af2802b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:37:15.002914 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5bb781b8-86cd-4dd2-a67f-12be8af2802b admin admin] The token cache is being invalidate because project e1464cdd99e64bbeab3d8f44b6b6e0ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:37:15.003123 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5bb781b8-86cd-4dd2-a67f-12be8af2802b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:37:15.003801 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-5bb781b8-86cd-4dd2-a67f-12be8af2802b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:37:15.006037 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1181/2362] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:37:14 2022] DELETE /identity/v3/projects/e1464cdd99e64bbeab3d8f44b6b6e0ab => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:37:22.853427 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d60ed7-1d23-45fe-a3b0-872f614123d5 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:37:22.854003 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d60ed7-1d23-45fe-a3b0-872f614123d5 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:37:22.854194 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d60ed7-1d23-45fe-a3b0-872f614123d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:37:22.871427 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-15d60ed7-1d23-45fe-a3b0-872f614123d5 None None] MFA Rules not processed for user `797a7b1e128f4102bac6430a60d2c0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:37:22.872621 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-15d60ed7-1d23-45fe-a3b0-872f614123d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:37:22.931719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1182/2363] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:37:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:37:24.999895 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:37:32.924342 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=41 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:38:00.760248 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb781b8-86cd-4dd2-a67f-12be8af2802b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:38:00.761770 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5bb781b8-86cd-4dd2-a67f-12be8af2802b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:38:00.819892 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb781b8-86cd-4dd2-a67f-12be8af2802b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:38:00.824276 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff1a2d3-6e85-48ee-9f05-2c6c576b8fba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:38:00.827351 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1a2d3-6e85-48ee-9f05-2c6c576b8fba service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:38:00.827351 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1a2d3-6e85-48ee-9f05-2c6c576b8fba service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:38:00.827351 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff1a2d3-6e85-48ee-9f05-2c6c576b8fba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:38:00.827351 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9ff1a2d3-6e85-48ee-9f05-2c6c576b8fba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:38:00.892040 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff1a2d3-6e85-48ee-9f05-2c6c576b8fba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:38:00.893500 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff1a2d3-6e85-48ee-9f05-2c6c576b8fba service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:38:00.903574 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1182/2364] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:38:00 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:38:10.898665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=50 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:38:19.501140 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a2aa8-3917-4007-bf17-5b8327108a6e None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:38:19.501696 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a2aa8-3917-4007-bf17-5b8327108a6e None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:38:19.501696 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a2aa8-3917-4007-bf17-5b8327108a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:38:19.519773 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-110a2aa8-3917-4007-bf17-5b8327108a6e None None] MFA Rules not processed for user `eecc2db7b5754624b678e51dd9a37a46`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:38:19.520985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-110a2aa8-3917-4007-bf17-5b8327108a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:38:19.574638 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1183/2365] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:38:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:38:28.087206 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff1a2d3-6e85-48ee-9f05-2c6c576b8fba service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:38:28.088725 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9ff1a2d3-6e85-48ee-9f05-2c6c576b8fba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:38:28.156515 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff1a2d3-6e85-48ee-9f05-2c6c576b8fba service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:38:28.161374 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f8a7aa-34b8-4f51-a281-2fb3d73a66bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:38:28.162241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8a7aa-34b8-4f51-a281-2fb3d73a66bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:38:28.163827 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8a7aa-34b8-4f51-a281-2fb3d73a66bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:38:28.163827 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f8a7aa-34b8-4f51-a281-2fb3d73a66bc admin admin] PATH_INFO: `/v3/users/4d368bff9f91483b81e6a192f7f90ff5` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:38:28.164029 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f8a7aa-34b8-4f51-a281-2fb3d73a66bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d368bff9f91483b81e6a192f7f90ff5)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:38:28.165994 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f8a7aa-34b8-4f51-a281-2fb3d73a66bc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:38:28.356587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-30f8a7aa-34b8-4f51-a281-2fb3d73a66bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d368bff9f91483b81e6a192f7f90ff5'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:38:28.357289 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-30f8a7aa-34b8-4f51-a281-2fb3d73a66bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d368bff9f91483b81e6a192f7f90ff5'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:38:28.364178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-30f8a7aa-34b8-4f51-a281-2fb3d73a66bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d368bff9f91483b81e6a192f7f90ff5'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:38:28.366378 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-30f8a7aa-34b8-4f51-a281-2fb3d73a66bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d368bff9f91483b81e6a192f7f90ff5'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:38:28.429232 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-30f8a7aa-34b8-4f51-a281-2fb3d73a66bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d368bff9f91483b81e6a192f7f90ff5'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:38:28.446578 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1183/2366] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:38:28 2022] DELETE /identity/v3/users/4d368bff9f91483b81e6a192f7f90ff5 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:38:28.688875 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a2aa8-3917-4007-bf17-5b8327108a6e None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:38:28.700107 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a2aa8-3917-4007-bf17-5b8327108a6e None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:38:28.757775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57e51d-f020-4198-bc32-265f72ca1ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:38:28.758641 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57e51d-f020-4198-bc32-265f72ca1ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:38:28.758927 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57e51d-f020-4198-bc32-265f72ca1ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:38:28.759207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf57e51d-f020-4198-bc32-265f72ca1ca0 admin admin] PATH_INFO: `/v3/projects/a80a4bd1d3764bb184ad239981d1e1f3` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:38:28.760426 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57e51d-f020-4198-bc32-265f72ca1ca0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a80a4bd1d3764bb184ad239981d1e1f3)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:38:28.762349 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf57e51d-f020-4198-bc32-265f72ca1ca0 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:38:28.862830 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bf57e51d-f020-4198-bc32-265f72ca1ca0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a80a4bd1d3764bb184ad239981d1e1f3', 'request_id': 'req-bf57e51d-f020-4198-bc32-265f72ca1ca0'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:38:28.868959 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bf57e51d-f020-4198-bc32-265f72ca1ca0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a80a4bd1d3764bb184ad239981d1e1f3', 'request_id': 'req-bf57e51d-f020-4198-bc32-265f72ca1ca0'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:38:28.875406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bf57e51d-f020-4198-bc32-265f72ca1ca0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a80a4bd1d3764bb184ad239981d1e1f3', 'request_id': 'req-bf57e51d-f020-4198-bc32-265f72ca1ca0'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:38:28.975889 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bf57e51d-f020-4198-bc32-265f72ca1ca0 admin admin] The token cache is being invalidate because project a80a4bd1d3764bb184ad239981d1e1f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:38:28.976101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bf57e51d-f020-4198-bc32-265f72ca1ca0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:38:28.976839 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bf57e51d-f020-4198-bc32-265f72ca1ca0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:38:28.978393 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1184/2367] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:38:28 2022] DELETE /identity/v3/projects/a80a4bd1d3764bb184ad239981d1e1f3 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:38:37.744798 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f8a7aa-34b8-4f51-a281-2fb3d73a66bc admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:38:37.747144 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-30f8a7aa-34b8-4f51-a281-2fb3d73a66bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:38:37.805859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f8a7aa-34b8-4f51-a281-2fb3d73a66bc admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:38:37.837509 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:38:37.853799 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc1946-f8b9-4b58-b01f-90ec9cd552aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:38:37.854449 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc1946-f8b9-4b58-b01f-90ec9cd552aa service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:38:37.854731 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc1946-f8b9-4b58-b01f-90ec9cd552aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:38:37.855005 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc1946-f8b9-4b58-b01f-90ec9cd552aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:38:37.856847 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-befc1946-f8b9-4b58-b01f-90ec9cd552aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:38:37.924481 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc1946-f8b9-4b58-b01f-90ec9cd552aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:38:37.926044 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc1946-f8b9-4b58-b01f-90ec9cd552aa service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:38:37.935898 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1184/2368] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:38:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:38:38.922104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57e51d-f020-4198-bc32-265f72ca1ca0 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:38:38.923817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-bf57e51d-f020-4198-bc32-265f72ca1ca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:38:38.965260 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:38:38.999753 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf57e51d-f020-4198-bc32-265f72ca1ca0 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:38:39.005744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cce85-7e42-4991-b17c-8354cf1a5ea2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:38:39.006380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cce85-7e42-4991-b17c-8354cf1a5ea2 service nova] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:38:39.006668 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cce85-7e42-4991-b17c-8354cf1a5ea2 service nova] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:38:39.006951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415cce85-7e42-4991-b17c-8354cf1a5ea2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:38:39.008852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-415cce85-7e42-4991-b17c-8354cf1a5ea2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:38:39.073608 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415cce85-7e42-4991-b17c-8354cf1a5ea2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:38:39.075321 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415cce85-7e42-4991-b17c-8354cf1a5ea2 service nova] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:38:39.086382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1185/2369] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:38:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:38:39.720817 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ea388-2f08-4605-b2db-ace2f308476c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:38:39.721313 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ea388-2f08-4605-b2db-ace2f308476c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:38:39.721419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ea388-2f08-4605-b2db-ace2f308476c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:38:39.740705 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0e0ea388-2f08-4605-b2db-ace2f308476c None None] MFA Rules not processed for user `c7f8abef711c4cd2b53852b3a969b120`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:38:39.741965 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0e0ea388-2f08-4605-b2db-ace2f308476c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:38:39.749326 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1185/2370] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:38:39 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:38:43.974933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cce85-7e42-4991-b17c-8354cf1a5ea2 service nova] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:38:43.976540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-415cce85-7e42-4991-b17c-8354cf1a5ea2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:38:43.988645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415cce85-7e42-4991-b17c-8354cf1a5ea2 service nova] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:38:43.992319 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4137c10-695e-4247-8e0c-fa6b6223e460 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:38:43.994442 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4137c10-695e-4247-8e0c-fa6b6223e460 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:38:43.994442 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4137c10-695e-4247-8e0c-fa6b6223e460 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:38:43.994442 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4137c10-695e-4247-8e0c-fa6b6223e460 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:38:43.995518 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-c4137c10-695e-4247-8e0c-fa6b6223e460 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:38:44.009728 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4137c10-695e-4247-8e0c-fa6b6223e460 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:38:44.011151 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4137c10-695e-4247-8e0c-fa6b6223e460 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:38:44.022836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1186/2371] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:38:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:38:46.508720 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0ea388-2f08-4605-b2db-ace2f308476c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:38:46.510104 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0e0ea388-2f08-4605-b2db-ace2f308476c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:38:46.522541 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0ea388-2f08-4605-b2db-ace2f308476c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:38:46.526100 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e4fc41-bb1c-4764-a826-47cfb1fb5fd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:38:46.526734 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4fc41-bb1c-4764-a826-47cfb1fb5fd6 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:38:46.527016 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4fc41-bb1c-4764-a826-47cfb1fb5fd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:38:46.527318 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e4fc41-bb1c-4764-a826-47cfb1fb5fd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:38:46.529036 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e1e4fc41-bb1c-4764-a826-47cfb1fb5fd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:38:46.542585 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e4fc41-bb1c-4764-a826-47cfb1fb5fd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:38:46.544036 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e4fc41-bb1c-4764-a826-47cfb1fb5fd6 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:38:46.555383 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1186/2372] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:38:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:38:54.017946 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=46 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:38:56.550639 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=41 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:39:01.243059 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5427fe3-6cb3-433a-9aee-9322de6b6393 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:01.243596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5427fe3-6cb3-433a-9aee-9322de6b6393 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:01.243694 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5427fe3-6cb3-433a-9aee-9322de6b6393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:01.260888 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-d5427fe3-6cb3-433a-9aee-9322de6b6393 None None] MFA Rules not processed for user `797a7b1e128f4102bac6430a60d2c0dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:39:01.262199 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d5427fe3-6cb3-433a-9aee-9322de6b6393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:01.273831 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1187/2373] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:39:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:39:01.575793 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e4fc41-bb1c-4764-a826-47cfb1fb5fd6 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:01.585317 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e4fc41-bb1c-4764-a826-47cfb1fb5fd6 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:01.589047 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04500c90-ff81-431f-8666-4a38658d9200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:01.589686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04500c90-ff81-431f-8666-4a38658d9200 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:01.589976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04500c90-ff81-431f-8666-4a38658d9200 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:01.590257 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04500c90-ff81-431f-8666-4a38658d9200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:01.592060 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-04500c90-ff81-431f-8666-4a38658d9200 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:01.601794 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04500c90-ff81-431f-8666-4a38658d9200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:01.603299 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04500c90-ff81-431f-8666-4a38658d9200 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:01.611584 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1187/2374] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:39:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:39:01.994723 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5427fe3-6cb3-433a-9aee-9322de6b6393 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:01.996249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d5427fe3-6cb3-433a-9aee-9322de6b6393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:02.071527 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5427fe3-6cb3-433a-9aee-9322de6b6393 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:02.076383 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ae642-afed-41f3-925f-27de87289025 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46c99f43a9344d7d92632990e45bb9cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:02.077033 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ae642-afed-41f3-925f-27de87289025 service placement] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:02.077317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ae642-afed-41f3-925f-27de87289025 service placement] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:02.077591 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ae642-afed-41f3-925f-27de87289025 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:02.079408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-669ae642-afed-41f3-925f-27de87289025 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:02.090597 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ae642-afed-41f3-925f-27de87289025 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:02.092213 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ae642-afed-41f3-925f-27de87289025 service placement] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:02.101947 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1188/2375] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:39:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:39:03.711818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04500c90-ff81-431f-8666-4a38658d9200 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:03.713192 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-04500c90-ff81-431f-8666-4a38658d9200 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:03.768799 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04500c90-ff81-431f-8666-4a38658d9200 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:03.772369 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c9539-b799-4483-b068-4ec303f7325c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:03.773007 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c9539-b799-4483-b068-4ec303f7325c service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:03.773292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c9539-b799-4483-b068-4ec303f7325c service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:03.773570 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c9539-b799-4483-b068-4ec303f7325c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:03.775346 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4d2c9539-b799-4483-b068-4ec303f7325c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:03.833386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c9539-b799-4483-b068-4ec303f7325c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:03.834868 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c9539-b799-4483-b068-4ec303f7325c service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:03.843882 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1188/2376] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:39:03 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:39:05.212200 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ae642-afed-41f3-925f-27de87289025 service placement] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:05.222614 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ae642-afed-41f3-925f-27de87289025 service placement] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:05.226113 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc937315-44cb-44ea-b10d-2af63fc35e72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:05.226745 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc937315-44cb-44ea-b10d-2af63fc35e72 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:05.227028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc937315-44cb-44ea-b10d-2af63fc35e72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:05.227326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc937315-44cb-44ea-b10d-2af63fc35e72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:05.229085 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-bc937315-44cb-44ea-b10d-2af63fc35e72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:05.239931 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc937315-44cb-44ea-b10d-2af63fc35e72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:05.241383 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc937315-44cb-44ea-b10d-2af63fc35e72 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:05.249718 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1189/2377] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:39:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:39:06.083114 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c9539-b799-4483-b068-4ec303f7325c service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:06.085281 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4d2c9539-b799-4483-b068-4ec303f7325c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:06.107812 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c9539-b799-4483-b068-4ec303f7325c service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:06.115046 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e6077-fb35-454c-b23f-e4e87c204c04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:06.116024 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e6077-fb35-454c-b23f-e4e87c204c04 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:06.116313 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e6077-fb35-454c-b23f-e4e87c204c04 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:06.116589 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e6077-fb35-454c-b23f-e4e87c204c04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:06.118414 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0d5e6077-fb35-454c-b23f-e4e87c204c04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:06.131603 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e6077-fb35-454c-b23f-e4e87c204c04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:06.133222 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e6077-fb35-454c-b23f-e4e87c204c04 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:06.145992 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1189/2378] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:39:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:39:08.205985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc937315-44cb-44ea-b10d-2af63fc35e72 service neutron] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:08.231255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc937315-44cb-44ea-b10d-2af63fc35e72 service neutron] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:08.245387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d3c72b-24b8-4ebb-9678-443c9b353b6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:08.246231 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3c72b-24b8-4ebb-9678-443c9b353b6d service nova] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:08.246661 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3c72b-24b8-4ebb-9678-443c9b353b6d service nova] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:08.247128 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d3c72b-24b8-4ebb-9678-443c9b353b6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:08.249855 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-53d3c72b-24b8-4ebb-9678-443c9b353b6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:08.272263 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d3c72b-24b8-4ebb-9678-443c9b353b6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:08.274621 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d3c72b-24b8-4ebb-9678-443c9b353b6d service nova] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:08.293214 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1190/2379] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:39:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:39:16.140143 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=60 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:39:18.285531 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=47 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:39:21.773374 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d0d47-6824-4d28-9a76-20665fc25995 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:21.773852 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d0d47-6824-4d28-9a76-20665fc25995 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:21.773959 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d0d47-6824-4d28-9a76-20665fc25995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:21.822101 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-544d0d47-6824-4d28-9a76-20665fc25995 None None] MFA Rules not processed for user `66ad8defb9b6453c96f1a8a7343695ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:39:21.823382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-544d0d47-6824-4d28-9a76-20665fc25995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:21.887900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1190/2380] 10.222.0.47 () {58 vars in 967 bytes} [Thu Jun 30 10:39:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:39:21.929832 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d3c72b-24b8-4ebb-9678-443c9b353b6d service nova] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:21.931240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-53d3c72b-24b8-4ebb-9678-443c9b353b6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:22.021471 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d3c72b-24b8-4ebb-9678-443c9b353b6d service nova] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:22.026625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef6f139-a90b-4d9f-a433-ee5d8f4c4733 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07de926cb8594b098f79111dab8a9cc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:22.027291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6f139-a90b-4d9f-a433-ee5d8f4c4733 service swift] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:22.027598 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6f139-a90b-4d9f-a433-ee5d8f4c4733 service swift] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:22.027875 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef6f139-a90b-4d9f-a433-ee5d8f4c4733 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:22.036673 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef6f139-a90b-4d9f-a433-ee5d8f4c4733 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:22.038368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef6f139-a90b-4d9f-a433-ee5d8f4c4733 service swift] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:22.046771 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1191/2381] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jun 30 10:39:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:39:26.756516 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d0d47-6824-4d28-9a76-20665fc25995 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:26.768875 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d0d47-6824-4d28-9a76-20665fc25995 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:26.774676 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9a566f-9d75-47ee-94d8-b247425d528e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46c99f43a9344d7d92632990e45bb9cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:26.775362 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a566f-9d75-47ee-94d8-b247425d528e service placement] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:26.775653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a566f-9d75-47ee-94d8-b247425d528e service placement] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:26.775961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9a566f-9d75-47ee-94d8-b247425d528e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:26.777937 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-aa9a566f-9d75-47ee-94d8-b247425d528e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:26.791519 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9a566f-9d75-47ee-94d8-b247425d528e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:26.793579 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9a566f-9d75-47ee-94d8-b247425d528e service placement] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:26.804845 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1191/2382] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:39:26 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:39:27.557805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef6f139-a90b-4d9f-a433-ee5d8f4c4733 service swift] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:27.567138 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef6f139-a90b-4d9f-a433-ee5d8f4c4733 service swift] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:27.570650 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a9cf9-b3cf-4dc0-a31b-b7f354ee9786 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:27.571404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a9cf9-b3cf-4dc0-a31b-b7f354ee9786 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:27.571688 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a9cf9-b3cf-4dc0-a31b-b7f354ee9786 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:27.571965 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69a9cf9-b3cf-4dc0-a31b-b7f354ee9786 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:27.573725 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-a69a9cf9-b3cf-4dc0-a31b-b7f354ee9786 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:27.639545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a9cf9-b3cf-4dc0-a31b-b7f354ee9786 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:27.640990 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69a9cf9-b3cf-4dc0-a31b-b7f354ee9786 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:27.650915 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1192/2383] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:39:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:39:32.104873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9a566f-9d75-47ee-94d8-b247425d528e service placement] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:32.106713 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-aa9a566f-9d75-47ee-94d8-b247425d528e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:32.124478 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9a566f-9d75-47ee-94d8-b247425d528e service placement] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:32.130069 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe5920-2c6d-4d43-a985-f962dd08c13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:32.131949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe5920-2c6d-4d43-a985-f962dd08c13b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:32.132458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe5920-2c6d-4d43-a985-f962dd08c13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:32.133443 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe5920-2c6d-4d43-a985-f962dd08c13b admin admin] PATH_INFO: `/v3/users/c9826b2f5d424025b8daff42e066419a` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:32.135701 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe5920-2c6d-4d43-a985-f962dd08c13b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9826b2f5d424025b8daff42e066419a)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:32.139250 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe5920-2c6d-4d43-a985-f962dd08c13b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:32.354088 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e5fe5920-2c6d-4d43-a985-f962dd08c13b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9826b2f5d424025b8daff42e066419a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:32.355003 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e5fe5920-2c6d-4d43-a985-f962dd08c13b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9826b2f5d424025b8daff42e066419a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:32.364072 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e5fe5920-2c6d-4d43-a985-f962dd08c13b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9826b2f5d424025b8daff42e066419a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:32.365157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e5fe5920-2c6d-4d43-a985-f962dd08c13b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9826b2f5d424025b8daff42e066419a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:32.484345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e5fe5920-2c6d-4d43-a985-f962dd08c13b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9826b2f5d424025b8daff42e066419a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:32.501015 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a9cf9-b3cf-4dc0-a31b-b7f354ee9786 service neutron] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:32.502589 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-a69a9cf9-b3cf-4dc0-a31b-b7f354ee9786 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:32.520872 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69a9cf9-b3cf-4dc0-a31b-b7f354ee9786 service neutron] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:32.521674 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1192/2384] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:39:32 2022] DELETE /identity/v3/users/c9826b2f5d424025b8daff42e066419a => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:39:32.585920 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48bb32-9faf-43c9-aac0-ef65a3b6fd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:32.586820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48bb32-9faf-43c9-aac0-ef65a3b6fd36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:32.587109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48bb32-9faf-43c9-aac0-ef65a3b6fd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:32.587486 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48bb32-9faf-43c9-aac0-ef65a3b6fd36 admin admin] PATH_INFO: `/v3/users/eecc2db7b5754624b678e51dd9a37a46` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:32.595048 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48bb32-9faf-43c9-aac0-ef65a3b6fd36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eecc2db7b5754624b678e51dd9a37a46)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:32.595048 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48bb32-9faf-43c9-aac0-ef65a3b6fd36 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:32.600870 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:39:32.778077 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe5920-2c6d-4d43-a985-f962dd08c13b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:32.778237 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e5fe5920-2c6d-4d43-a985-f962dd08c13b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:32.795475 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe5920-2c6d-4d43-a985-f962dd08c13b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:32.808959 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b1413-4c60-4c74-9cb8-be3b004e6967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:32.809892 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b1413-4c60-4c74-9cb8-be3b004e6967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:32.810200 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b1413-4c60-4c74-9cb8-be3b004e6967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:32.810552 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b1413-4c60-4c74-9cb8-be3b004e6967 admin admin] PATH_INFO: `/v3/projects/ff8c3b3c0caa43d4b75ae94fc89c5568` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:32.811707 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b1413-4c60-4c74-9cb8-be3b004e6967 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff8c3b3c0caa43d4b75ae94fc89c5568)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:32.813761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b1413-4c60-4c74-9cb8-be3b004e6967 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:32.853688 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2f48bb32-9faf-43c9-aac0-ef65a3b6fd36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eecc2db7b5754624b678e51dd9a37a46'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:32.924086 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2f48bb32-9faf-43c9-aac0-ef65a3b6fd36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eecc2db7b5754624b678e51dd9a37a46'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:32.926215 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:39:32.940218 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2f48bb32-9faf-43c9-aac0-ef65a3b6fd36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eecc2db7b5754624b678e51dd9a37a46'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:32.941754 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2f48bb32-9faf-43c9-aac0-ef65a3b6fd36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eecc2db7b5754624b678e51dd9a37a46'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:32.942729 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2f48bb32-9faf-43c9-aac0-ef65a3b6fd36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eecc2db7b5754624b678e51dd9a37a46'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:32.945746 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-981b1413-4c60-4c74-9cb8-be3b004e6967 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff8c3b3c0caa43d4b75ae94fc89c5568', 'request_id': 'req-981b1413-4c60-4c74-9cb8-be3b004e6967'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:32.957999 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1193/2385] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:39:32 2022] DELETE /identity/v3/users/eecc2db7b5754624b678e51dd9a37a46 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:39:33.017645 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-981b1413-4c60-4c74-9cb8-be3b004e6967 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff8c3b3c0caa43d4b75ae94fc89c5568', 'request_id': 'req-981b1413-4c60-4c74-9cb8-be3b004e6967'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:33.024573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-981b1413-4c60-4c74-9cb8-be3b004e6967 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff8c3b3c0caa43d4b75ae94fc89c5568', 'request_id': 'req-981b1413-4c60-4c74-9cb8-be3b004e6967'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:33.036286 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-981b1413-4c60-4c74-9cb8-be3b004e6967 admin admin] The token cache is being invalidate because project ff8c3b3c0caa43d4b75ae94fc89c5568 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:39:33.036579 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-981b1413-4c60-4c74-9cb8-be3b004e6967 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:33.037238 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-981b1413-4c60-4c74-9cb8-be3b004e6967 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:33.038889 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1193/2386] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:39:32 2022] DELETE /identity/v3/projects/ff8c3b3c0caa43d4b75ae94fc89c5568 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:39:33.044634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48bb32-9faf-43c9-aac0-ef65a3b6fd36 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:33.046063 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-2f48bb32-9faf-43c9-aac0-ef65a3b6fd36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:33.127193 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48bb32-9faf-43c9-aac0-ef65a3b6fd36 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:33.134059 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d8e2f-f247-4ef7-905f-ee5306ddb366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:33.134983 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d8e2f-f247-4ef7-905f-ee5306ddb366 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:33.135286 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d8e2f-f247-4ef7-905f-ee5306ddb366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:33.135572 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d8e2f-f247-4ef7-905f-ee5306ddb366 admin admin] PATH_INFO: `/v3/users/ef14698bfec847039304f3a016238059` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:33.136834 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d8e2f-f247-4ef7-905f-ee5306ddb366 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef14698bfec847039304f3a016238059)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:33.138802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d8e2f-f247-4ef7-905f-ee5306ddb366 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:33.246316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b1413-4c60-4c74-9cb8-be3b004e6967 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:33.248364 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-981b1413-4c60-4c74-9cb8-be3b004e6967 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:33.319700 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b1413-4c60-4c74-9cb8-be3b004e6967 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:33.324308 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a072fb-ca7b-4a81-ac62-32f782e428e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:33.325168 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a072fb-ca7b-4a81-ac62-32f782e428e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:33.325453 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a072fb-ca7b-4a81-ac62-32f782e428e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:33.325728 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a072fb-ca7b-4a81-ac62-32f782e428e5 admin admin] PATH_INFO: `/v3/projects/c1955ca75a3b42d28474c3d7df74c9a6` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:33.326909 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a072fb-ca7b-4a81-ac62-32f782e428e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1955ca75a3b42d28474c3d7df74c9a6)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:33.328877 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a072fb-ca7b-4a81-ac62-32f782e428e5 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:33.391890 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d32d8e2f-f247-4ef7-905f-ee5306ddb366 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef14698bfec847039304f3a016238059'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:33.451803 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d2a072fb-ca7b-4a81-ac62-32f782e428e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1955ca75a3b42d28474c3d7df74c9a6', 'request_id': 'req-d2a072fb-ca7b-4a81-ac62-32f782e428e5'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:33.463125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d32d8e2f-f247-4ef7-905f-ee5306ddb366 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef14698bfec847039304f3a016238059'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:33.477741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d32d8e2f-f247-4ef7-905f-ee5306ddb366 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef14698bfec847039304f3a016238059'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:33.478458 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d32d8e2f-f247-4ef7-905f-ee5306ddb366 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef14698bfec847039304f3a016238059'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:33.479071 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d32d8e2f-f247-4ef7-905f-ee5306ddb366 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef14698bfec847039304f3a016238059'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:33.491843 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1194/2387] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:39:33 2022] DELETE /identity/v3/users/ef14698bfec847039304f3a016238059 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:39:33.564146 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d2a072fb-ca7b-4a81-ac62-32f782e428e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1955ca75a3b42d28474c3d7df74c9a6', 'request_id': 'req-d2a072fb-ca7b-4a81-ac62-32f782e428e5'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:33.570436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d2a072fb-ca7b-4a81-ac62-32f782e428e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1955ca75a3b42d28474c3d7df74c9a6', 'request_id': 'req-d2a072fb-ca7b-4a81-ac62-32f782e428e5'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:33.581649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d2a072fb-ca7b-4a81-ac62-32f782e428e5 admin admin] The token cache is being invalidate because project c1955ca75a3b42d28474c3d7df74c9a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:39:33.581934 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d2a072fb-ca7b-4a81-ac62-32f782e428e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:33.582579 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d2a072fb-ca7b-4a81-ac62-32f782e428e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:33.584160 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1194/2388] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:39:33 2022] DELETE /identity/v3/projects/c1955ca75a3b42d28474c3d7df74c9a6 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:39:33.594434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e2d63-1c9d-42cf-8d84-1c7fd26ef788 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:33.594915 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e2d63-1c9d-42cf-8d84-1c7fd26ef788 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:33.595034 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e2d63-1c9d-42cf-8d84-1c7fd26ef788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:33.602075 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:39:33.616376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-a15e2d63-1c9d-42cf-8d84-1c7fd26ef788 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:39:33.617670 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-a15e2d63-1c9d-42cf-8d84-1c7fd26ef788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:33.678113 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1195/2389] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:39:33.686353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e7c6d-b633-40fe-852e-817d0ebf73a1 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:33.686640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e7c6d-b633-40fe-852e-817d0ebf73a1 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:33.686916 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7e7c6d-b633-40fe-852e-817d0ebf73a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:33.706076 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0b7e7c6d-b633-40fe-852e-817d0ebf73a1 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:39:33.707981 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0b7e7c6d-b633-40fe-852e-817d0ebf73a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:33.720424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1195/2390] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:39:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:39:33.721076 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e2d63-1c9d-42cf-8d84-1c7fd26ef788 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:33.734426 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e2d63-1c9d-42cf-8d84-1c7fd26ef788 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:33.736662 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d50167-8f11-47f4-ab3f-7cf9a3e5e052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:33.737331 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d50167-8f11-47f4-ab3f-7cf9a3e5e052 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:33.738493 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d50167-8f11-47f4-ab3f-7cf9a3e5e052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:33.738718 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e7c6d-b633-40fe-852e-817d0ebf73a1 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:33.739135 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d50167-8f11-47f4-ab3f-7cf9a3e5e052 admin admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:33.739902 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d50167-8f11-47f4-ab3f-7cf9a3e5e052 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:39:33.740211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d50167-8f11-47f4-ab3f-7cf9a3e5e052 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:33.741762 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0b7e7c6d-b633-40fe-852e-817d0ebf73a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:33.741918 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d50167-8f11-47f4-ab3f-7cf9a3e5e052 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:33.759073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1196/2391] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:39:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:39:33.770229 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7e7c6d-b633-40fe-852e-817d0ebf73a1 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:33.773651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d50167-8f11-47f4-ab3f-7cf9a3e5e052 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:33.775041 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45d22fe-00a7-4a20-aa7e-32a6c4813c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:33.775951 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d22fe-00a7-4a20-aa7e-32a6c4813c79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:33.776239 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d22fe-00a7-4a20-aa7e-32a6c4813c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:33.776626 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45d22fe-00a7-4a20-aa7e-32a6c4813c79 admin admin] PATH_INFO: `/v3/projects/4d0dd3b46456410183c7a1f23069e964` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:33.777957 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d22fe-00a7-4a20-aa7e-32a6c4813c79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d0dd3b46456410183c7a1f23069e964)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:33.779943 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45d22fe-00a7-4a20-aa7e-32a6c4813c79 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:33.783747 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d50167-8f11-47f4-ab3f-7cf9a3e5e052 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:33.790570 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa14d1ab-c036-4aef-aab6-13bf44dccc2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:33.791338 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14d1ab-c036-4aef-aab6-13bf44dccc2b admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:33.795547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14d1ab-c036-4aef-aab6-13bf44dccc2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:33.795547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa14d1ab-c036-4aef-aab6-13bf44dccc2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:33.795547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa14d1ab-c036-4aef-aab6-13bf44dccc2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:33.795547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa14d1ab-c036-4aef-aab6-13bf44dccc2b admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:33.897138 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:39:33.910038 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d45d22fe-00a7-4a20-aa7e-32a6c4813c79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d0dd3b46456410183c7a1f23069e964', 'request_id': 'req-d45d22fe-00a7-4a20-aa7e-32a6c4813c79'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:33.931715 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1197/2392] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:39:33 2022] POST /identity/v3/projects => generated 389 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:39:33.935441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa14d1ab-c036-4aef-aab6-13bf44dccc2b admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:33.996259 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa14d1ab-c036-4aef-aab6-13bf44dccc2b admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:34.000849 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c01d063-f186-449e-88d1-48fec80ba5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:34.001521 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01d063-f186-449e-88d1-48fec80ba5a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:34.001810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01d063-f186-449e-88d1-48fec80ba5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:34.002090 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c01d063-f186-449e-88d1-48fec80ba5a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:34.002817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01d063-f186-449e-88d1-48fec80ba5a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:34.004572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d45d22fe-00a7-4a20-aa7e-32a6c4813c79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d0dd3b46456410183c7a1f23069e964', 'request_id': 'req-d45d22fe-00a7-4a20-aa7e-32a6c4813c79'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:34.005568 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c01d063-f186-449e-88d1-48fec80ba5a6 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:34.010722 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d45d22fe-00a7-4a20-aa7e-32a6c4813c79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d0dd3b46456410183c7a1f23069e964', 'request_id': 'req-d45d22fe-00a7-4a20-aa7e-32a6c4813c79'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:34.022316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d45d22fe-00a7-4a20-aa7e-32a6c4813c79 admin admin] The token cache is being invalidate because project 4d0dd3b46456410183c7a1f23069e964 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:39:34.027844 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d45d22fe-00a7-4a20-aa7e-32a6c4813c79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:34.027844 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d45d22fe-00a7-4a20-aa7e-32a6c4813c79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:39:34.027844 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1196/2393] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:39:33 2022] DELETE /identity/v3/projects/4d0dd3b46456410183c7a1f23069e964 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:39:34.029718 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93051055-36d2-4f62-b86d-dd25b140524d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:34.030005 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93051055-36d2-4f62-b86d-dd25b140524d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:34.030288 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93051055-36d2-4f62-b86d-dd25b140524d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:34.047913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-93051055-36d2-4f62-b86d-dd25b140524d None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:39:34.049108 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-93051055-36d2-4f62-b86d-dd25b140524d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:34.057561 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1197/2394] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:39:34.115917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1198/2395] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:39:33 2022] POST /identity/v3/users => generated 295 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:39:34.120225 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93051055-36d2-4f62-b86d-dd25b140524d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:34.121677 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-93051055-36d2-4f62-b86d-dd25b140524d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:34.128074 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640e836-48c3-450c-aa40-fda8d2d21587 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:34.128895 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640e836-48c3-450c-aa40-fda8d2d21587 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:34.129178 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3640e836-48c3-450c-aa40-fda8d2d21587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:34.134252 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93051055-36d2-4f62-b86d-dd25b140524d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:34.139082 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19c15e-5b20-4c1f-865b-acd7459c5470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:34.139783 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19c15e-5b20-4c1f-865b-acd7459c5470 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:34.140069 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19c15e-5b20-4c1f-865b-acd7459c5470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:34.140344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19c15e-5b20-4c1f-865b-acd7459c5470 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:34.141073 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac19c15e-5b20-4c1f-865b-acd7459c5470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:34.142767 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac19c15e-5b20-4c1f-865b-acd7459c5470 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:34.148038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-3640e836-48c3-450c-aa40-fda8d2d21587 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:39:34.149237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-3640e836-48c3-450c-aa40-fda8d2d21587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:34.153491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1198/2396] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:39:34.157523 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1199/2397] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:39:34.157648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19c15e-5b20-4c1f-865b-acd7459c5470 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:34.162900 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640e836-48c3-450c-aa40-fda8d2d21587 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:34.168327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19c15e-5b20-4c1f-865b-acd7459c5470 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:34.171766 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3640e836-48c3-450c-aa40-fda8d2d21587 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:34.173268 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a98878-6574-4057-9da3-3d3e19fd74b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:34.173926 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a98878-6574-4057-9da3-3d3e19fd74b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:34.174212 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a98878-6574-4057-9da3-3d3e19fd74b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:34.174491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a98878-6574-4057-9da3-3d3e19fd74b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:34.175750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a98878-6574-4057-9da3-3d3e19fd74b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:34.177039 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a98878-6574-4057-9da3-3d3e19fd74b1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:34.178376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaaba4f-c0e8-43ca-a708-a1e92187875e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:34.179191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaaba4f-c0e8-43ca-a708-a1e92187875e admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:34.179646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaaba4f-c0e8-43ca-a708-a1e92187875e admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:34.180069 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaaba4f-c0e8-43ca-a708-a1e92187875e admin admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:34.181763 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaaba4f-c0e8-43ca-a708-a1e92187875e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:39:34.181763 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaaba4f-c0e8-43ca-a708-a1e92187875e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:34.183139 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaaba4f-c0e8-43ca-a708-a1e92187875e admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:34.187648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1199/2398] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:39:34.191662 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a98878-6574-4057-9da3-3d3e19fd74b1 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:34.195974 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1200/2399] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:39:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:39:34.200270 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaaba4f-c0e8-43ca-a708-a1e92187875e admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:34.202213 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a98878-6574-4057-9da3-3d3e19fd74b1 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:34.207456 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c002e-5faa-49f8-aab8-82406f6ad0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:34.208581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c002e-5faa-49f8-aab8-82406f6ad0d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:34.208873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c002e-5faa-49f8-aab8-82406f6ad0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:34.209146 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c002e-5faa-49f8-aab8-82406f6ad0d7 admin admin] PATH_INFO: `/v3/projects/3b23261b1f1944d4a3bbe630bcde4d61/users/1f4a95c82152441096c50f79acc68e9a/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:34.211226 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaaba4f-c0e8-43ca-a708-a1e92187875e admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:34.216960 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b821d-05a3-4e2a-a707-c936d7fab320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:34.221625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b821d-05a3-4e2a-a707-c936d7fab320 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:34.221625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b821d-05a3-4e2a-a707-c936d7fab320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:34.221625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b821d-05a3-4e2a-a707-c936d7fab320 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:34.221625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b821d-05a3-4e2a-a707-c936d7fab320 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:34.221625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b821d-05a3-4e2a-a707-c936d7fab320 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:34.225637 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c002e-5faa-49f8-aab8-82406f6ad0d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b23261b1f1944d4a3bbe630bcde4d61, user_id=1f4a95c82152441096c50f79acc68e9a, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:34.228455 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6c002e-5faa-49f8-aab8-82406f6ad0d7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:34.292242 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1200/2400] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:39:34 2022] PUT /identity/v3/projects/3b23261b1f1944d4a3bbe630bcde4d61/users/1f4a95c82152441096c50f79acc68e9a/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:39:34.293755 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1201/2401] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:39:34 2022] POST /identity/v3/projects => generated 379 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:39:34.297170 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f2ff3-825f-471e-87b4-77f1a6e2d9be None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:34.297461 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f2ff3-825f-471e-87b4-77f1a6e2d9be None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:34.297741 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f2ff3-825f-471e-87b4-77f1a6e2d9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:34.298033 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b821d-05a3-4e2a-a707-c936d7fab320 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:34.315137 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:39:34.316878 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-249f2ff3-825f-471e-87b4-77f1a6e2d9be None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:39:34.318109 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-249f2ff3-825f-471e-87b4-77f1a6e2d9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:34.359793 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b821d-05a3-4e2a-a707-c936d7fab320 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:34.364929 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c87d46-b12b-4539-a49c-2645a6c0736a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:34.365681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c87d46-b12b-4539-a49c-2645a6c0736a admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:34.366062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c87d46-b12b-4539-a49c-2645a6c0736a admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:34.366349 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c87d46-b12b-4539-a49c-2645a6c0736a admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:34.367069 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c87d46-b12b-4539-a49c-2645a6c0736a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:34.369043 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c87d46-b12b-4539-a49c-2645a6c0736a admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:34.376973 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1201/2402] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:39:34.387624 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f2ff3-825f-471e-87b4-77f1a6e2d9be None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:34.392224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-249f2ff3-825f-471e-87b4-77f1a6e2d9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:34.444640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f2ff3-825f-471e-87b4-77f1a6e2d9be None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:34.448280 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e6062-a21a-45ff-a652-4bdf4ba7eb0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:34.448929 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e6062-a21a-45ff-a652-4bdf4ba7eb0f service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:34.449216 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e6062-a21a-45ff-a652-4bdf4ba7eb0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:34.449498 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e6062-a21a-45ff-a652-4bdf4ba7eb0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:34.459724 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7e6062-a21a-45ff-a652-4bdf4ba7eb0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:34.461918 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7e6062-a21a-45ff-a652-4bdf4ba7eb0f service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:34.472423 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1202/2403] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:39:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:39:34.488344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1202/2404] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:39:34 2022] POST /identity/v3/users => generated 290 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:39:34.492534 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e6062-a21a-45ff-a652-4bdf4ba7eb0f service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:34.502374 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7e6062-a21a-45ff-a652-4bdf4ba7eb0f service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:34.507709 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6e8b5-813c-4233-af53-d2d050d260f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:34.508378 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6e8b5-813c-4233-af53-d2d050d260f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:34.508694 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6e8b5-813c-4233-af53-d2d050d260f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:34.508973 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd6e8b5-813c-4233-af53-d2d050d260f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:34.509705 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd6e8b5-813c-4233-af53-d2d050d260f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:34.511348 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd6e8b5-813c-4233-af53-d2d050d260f6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:34.519156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:39:34.524031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1203/2405] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:39:34.528185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c87d46-b12b-4539-a49c-2645a6c0736a admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:34.538735 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c87d46-b12b-4539-a49c-2645a6c0736a admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:34.543530 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ed901-8e21-403a-8795-d27f4ce32dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:34.544132 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ed901-8e21-403a-8795-d27f4ce32dae admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:34.544419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ed901-8e21-403a-8795-d27f4ce32dae admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:34.544699 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ed901-8e21-403a-8795-d27f4ce32dae admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:34.545422 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ed901-8e21-403a-8795-d27f4ce32dae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:34.547015 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ed901-8e21-403a-8795-d27f4ce32dae admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:34.557451 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1203/2406] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:39:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:39:34.561492 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6e8b5-813c-4233-af53-d2d050d260f6 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:34.571686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd6e8b5-813c-4233-af53-d2d050d260f6 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:34.576312 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7b09f-850e-4b9b-9c3e-32c06253c942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:34.577188 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7b09f-850e-4b9b-9c3e-32c06253c942 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:34.581562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7b09f-850e-4b9b-9c3e-32c06253c942 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:34.581562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae7b09f-850e-4b9b-9c3e-32c06253c942 admin admin] PATH_INFO: `/v3/projects/158457f258274015ba394ce8cfd86a9a/users/fa58de3ae9eb4dc9bd80fc41ec3e0793/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:34.592046 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7b09f-850e-4b9b-9c3e-32c06253c942 admin admin] RBAC: Authorizing `identity:create_grant(project_id=158457f258274015ba394ce8cfd86a9a, user_id=fa58de3ae9eb4dc9bd80fc41ec3e0793, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:34.595035 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae7b09f-850e-4b9b-9c3e-32c06253c942 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:34.697036 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1204/2407] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:39:34 2022] PUT /identity/v3/projects/158457f258274015ba394ce8cfd86a9a/users/fa58de3ae9eb4dc9bd80fc41ec3e0793/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:39:34.702282 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee8f8b-de60-4195-89ac-092e6124e51c None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:34.702859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee8f8b-de60-4195-89ac-092e6124e51c None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:34.702859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee8f8b-de60-4195-89ac-092e6124e51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:34.721619 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-17ee8f8b-de60-4195-89ac-092e6124e51c None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:39:34.723049 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-17ee8f8b-de60-4195-89ac-092e6124e51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:34.787214 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1204/2408] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:39:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:39:34.797639 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7b09f-850e-4b9b-9c3e-32c06253c942 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:34.799342 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5ae7b09f-850e-4b9b-9c3e-32c06253c942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:34.872572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae7b09f-850e-4b9b-9c3e-32c06253c942 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:34.876889 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca81518-83b9-4709-8bd6-e1033f6c758a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:34.877799 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca81518-83b9-4709-8bd6-e1033f6c758a service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:34.878838 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca81518-83b9-4709-8bd6-e1033f6c758a service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:34.879456 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca81518-83b9-4709-8bd6-e1033f6c758a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:34.892249 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca81518-83b9-4709-8bd6-e1033f6c758a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:34.894636 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca81518-83b9-4709-8bd6-e1033f6c758a service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:34.905886 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1205/2409] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:39:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:39:41.064653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69538571-ce45-436f-ad31-151778bc5433 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:41.066191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69538571-ce45-436f-ad31-151778bc5433 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:41.066460 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69538571-ce45-436f-ad31-151778bc5433 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:41.097044 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-69538571-ce45-436f-ad31-151778bc5433 None None] MFA Rules not processed for user `1f4a95c82152441096c50f79acc68e9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:39:41.099071 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-69538571-ce45-436f-ad31-151778bc5433 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:41.222467 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1205/2410] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:39:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:39:41.241802 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca81518-83b9-4709-8bd6-e1033f6c758a service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:41.243550 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0ca81518-83b9-4709-8bd6-e1033f6c758a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:41.430282 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca81518-83b9-4709-8bd6-e1033f6c758a service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:41.454742 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06da01-df2b-40ca-8d77-227f67bd45dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:41.458446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06da01-df2b-40ca-8d77-227f67bd45dd service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:41.458446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06da01-df2b-40ca-8d77-227f67bd45dd service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:41.458446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa06da01-df2b-40ca-8d77-227f67bd45dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:41.468693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa06da01-df2b-40ca-8d77-227f67bd45dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:41.473954 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa06da01-df2b-40ca-8d77-227f67bd45dd service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:41.500468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1206/2411] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:39:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:39:42.021049 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f457aa1-4d34-492f-86df-b332780fb3f3 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:42.021979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f457aa1-4d34-492f-86df-b332780fb3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:42.021979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f457aa1-4d34-492f-86df-b332780fb3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:42.073127 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-8f457aa1-4d34-492f-86df-b332780fb3f3 None None] MFA Rules not processed for user `fa58de3ae9eb4dc9bd80fc41ec3e0793`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:39:42.075115 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-8f457aa1-4d34-492f-86df-b332780fb3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:42.106942 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:39:42.177912 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1206/2412] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:39:42 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:39:42.182530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06da01-df2b-40ca-8d77-227f67bd45dd service nova] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:42.198774 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa06da01-df2b-40ca-8d77-227f67bd45dd service nova] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:42.202727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa582e-598b-4047-86fc-0e9f6964ca20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:42.203406 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa582e-598b-4047-86fc-0e9f6964ca20 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:42.203692 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa582e-598b-4047-86fc-0e9f6964ca20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:42.203972 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fa582e-598b-4047-86fc-0e9f6964ca20 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:42.204951 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa582e-598b-4047-86fc-0e9f6964ca20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:42.207735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fa582e-598b-4047-86fc-0e9f6964ca20 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:42.321419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1207/2413] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:39:42 2022] POST /identity/v3/projects => generated 377 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:39:42.326051 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f457aa1-4d34-492f-86df-b332780fb3f3 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:42.337931 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f457aa1-4d34-492f-86df-b332780fb3f3 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:42.441568 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6026809d-4709-4815-95df-de1cdfa9474d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:42.442264 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6026809d-4709-4815-95df-de1cdfa9474d admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:42.442643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6026809d-4709-4815-95df-de1cdfa9474d admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:42.442810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6026809d-4709-4815-95df-de1cdfa9474d admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:42.444354 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6026809d-4709-4815-95df-de1cdfa9474d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:42.446513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6026809d-4709-4815-95df-de1cdfa9474d admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:42.578355 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1207/2414] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:39:42 2022] POST /identity/v3/users => generated 295 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:39:42.584089 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa582e-598b-4047-86fc-0e9f6964ca20 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:42.606202 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fa582e-598b-4047-86fc-0e9f6964ca20 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:42.621006 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed71ea4c-2fda-4d05-9776-f537fb5a653a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:42.621796 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71ea4c-2fda-4d05-9776-f537fb5a653a admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:42.622301 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71ea4c-2fda-4d05-9776-f537fb5a653a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:42.622814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed71ea4c-2fda-4d05-9776-f537fb5a653a admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:42.623895 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed71ea4c-2fda-4d05-9776-f537fb5a653a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:42.625758 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed71ea4c-2fda-4d05-9776-f537fb5a653a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:42.640651 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1208/2415] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:39:42.643167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6026809d-4709-4815-95df-de1cdfa9474d admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:42.655355 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6026809d-4709-4815-95df-de1cdfa9474d admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:42.666083 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff85ff-18e8-4fce-9df2-71b2188938d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:42.667004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff85ff-18e8-4fce-9df2-71b2188938d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:42.667526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff85ff-18e8-4fce-9df2-71b2188938d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:42.667817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eff85ff-18e8-4fce-9df2-71b2188938d8 admin admin] PATH_INFO: `/v3/projects/2fa2b32a2a5145f786d18ab7218a9cf6/users/1bea5ab441114aea884879410a735631/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:42.694893 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eff85ff-18e8-4fce-9df2-71b2188938d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fa2b32a2a5145f786d18ab7218a9cf6, user_id=1bea5ab441114aea884879410a735631, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:42.697960 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eff85ff-18e8-4fce-9df2-71b2188938d8 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:42.778408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1208/2416] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:39:42 2022] PUT /identity/v3/projects/2fa2b32a2a5145f786d18ab7218a9cf6/users/1bea5ab441114aea884879410a735631/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:39:42.783782 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed71ea4c-2fda-4d05-9776-f537fb5a653a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:42.794346 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed71ea4c-2fda-4d05-9776-f537fb5a653a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:42.818508 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:39:42.867785 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822120aa-b9cc-4bc1-8c6c-c5378a9e2cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:42.868474 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822120aa-b9cc-4bc1-8c6c-c5378a9e2cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:42.868764 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822120aa-b9cc-4bc1-8c6c-c5378a9e2cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:42.869045 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822120aa-b9cc-4bc1-8c6c-c5378a9e2cb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:42.869776 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822120aa-b9cc-4bc1-8c6c-c5378a9e2cb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:42.871440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822120aa-b9cc-4bc1-8c6c-c5378a9e2cb5 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:42.886612 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1209/2417] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:39:42.887660 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff85ff-18e8-4fce-9df2-71b2188938d8 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:42.901596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eff85ff-18e8-4fce-9df2-71b2188938d8 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:42.907205 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c9e048-0f3c-4c6b-b7d1-b06fb1b0aee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:42.908316 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9e048-0f3c-4c6b-b7d1-b06fb1b0aee7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:42.908646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9e048-0f3c-4c6b-b7d1-b06fb1b0aee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:42.908920 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c9e048-0f3c-4c6b-b7d1-b06fb1b0aee7 admin admin] PATH_INFO: `/v3/projects/2fa2b32a2a5145f786d18ab7218a9cf6/users/1bea5ab441114aea884879410a735631/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:42.911443 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c9e048-0f3c-4c6b-b7d1-b06fb1b0aee7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fa2b32a2a5145f786d18ab7218a9cf6, user_id=1bea5ab441114aea884879410a735631, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:42.914117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c9e048-0f3c-4c6b-b7d1-b06fb1b0aee7 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:42.941030 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1209/2418] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:39:42 2022] PUT /identity/v3/projects/2fa2b32a2a5145f786d18ab7218a9cf6/users/1bea5ab441114aea884879410a735631/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:39:44.606031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506ae5a-3c81-42c2-8a61-dd42677f4f23 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:44.606031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506ae5a-3c81-42c2-8a61-dd42677f4f23 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:44.606031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506ae5a-3c81-42c2-8a61-dd42677f4f23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:44.625265 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c506ae5a-3c81-42c2-8a61-dd42677f4f23 None None] MFA Rules not processed for user `1f4a95c82152441096c50f79acc68e9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:39:44.632087 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c506ae5a-3c81-42c2-8a61-dd42677f4f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:44.758825 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1210/2419] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:39:44 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:39:50.084261 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5e80a-f5a0-4dc5-ab55-7561b239ced7 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:50.084815 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5e80a-f5a0-4dc5-ab55-7561b239ced7 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:50.084815 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f5e80a-f5a0-4dc5-ab55-7561b239ced7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:50.129859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-78f5e80a-f5a0-4dc5-ab55-7561b239ced7 None None] MFA Rules not processed for user `1bea5ab441114aea884879410a735631`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:39:50.131368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-78f5e80a-f5a0-4dc5-ab55-7561b239ced7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:50.195187 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:39:50.231165 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1210/2420] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:39:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:39:50.240079 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506ae5a-3c81-42c2-8a61-dd42677f4f23 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:50.243028 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c506ae5a-3c81-42c2-8a61-dd42677f4f23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:50.390690 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c506ae5a-3c81-42c2-8a61-dd42677f4f23 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:50.403924 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1167732-1ba7-4b39-9f13-fcee30f6e121 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:50.405000 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1167732-1ba7-4b39-9f13-fcee30f6e121 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:50.405517 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1167732-1ba7-4b39-9f13-fcee30f6e121 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:50.405990 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1167732-1ba7-4b39-9f13-fcee30f6e121 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:50.476317 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1167732-1ba7-4b39-9f13-fcee30f6e121 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:50.478934 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1167732-1ba7-4b39-9f13-fcee30f6e121 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:50.493795 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=100 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:39:50.493795 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1211/2421] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:39:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:39:57.309302 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5e80a-f5a0-4dc5-ab55-7561b239ced7 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:39:57.324195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f5e80a-f5a0-4dc5-ab55-7561b239ced7 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:39:57.329704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555dc70a-d2fb-4267-8dd3-5563abb1df62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:39:57.330391 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555dc70a-d2fb-4267-8dd3-5563abb1df62 service nova] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:39:57.331580 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555dc70a-d2fb-4267-8dd3-5563abb1df62 service nova] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:39:57.331580 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555dc70a-d2fb-4267-8dd3-5563abb1df62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:39:57.332883 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-555dc70a-d2fb-4267-8dd3-5563abb1df62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:39:57.345189 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555dc70a-d2fb-4267-8dd3-5563abb1df62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:39:57.346841 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555dc70a-d2fb-4267-8dd3-5563abb1df62 service nova] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:39:57.357771 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1211/2422] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:39:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:40:07.353050 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=10 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:09.381970 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b4e4e-d972-4d72-913e-ee36efe06a5c None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:09.382563 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b4e4e-d972-4d72-913e-ee36efe06a5c None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:09.382563 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2b4e4e-d972-4d72-913e-ee36efe06a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:09.400534 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6a2b4e4e-d972-4d72-913e-ee36efe06a5c None None] MFA Rules not processed for user `fa58de3ae9eb4dc9bd80fc41ec3e0793`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:09.401760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6a2b4e4e-d972-4d72-913e-ee36efe06a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:09.409158 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1212/2423] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:09.755582 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555dc70a-d2fb-4267-8dd3-5563abb1df62 service nova] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:09.815079 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555dc70a-d2fb-4267-8dd3-5563abb1df62 service nova] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:09.818913 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e586b23-d1f3-40f6-9ce1-9f22b8e2b42b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:09.819587 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e586b23-d1f3-40f6-9ce1-9f22b8e2b42b service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:09.819871 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e586b23-d1f3-40f6-9ce1-9f22b8e2b42b service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:09.820149 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e586b23-d1f3-40f6-9ce1-9f22b8e2b42b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:09.821897 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-1e586b23-d1f3-40f6-9ce1-9f22b8e2b42b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:09.831962 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e586b23-d1f3-40f6-9ce1-9f22b8e2b42b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:09.833476 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e586b23-d1f3-40f6-9ce1-9f22b8e2b42b service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:09.838436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b4e4e-d972-4d72-913e-ee36efe06a5c None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:09.841752 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1212/2424] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:40:09.849617 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2b4e4e-d972-4d72-913e-ee36efe06a5c None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:09.853286 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756b939-0f66-4267-8d15-a53257ec3893 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:09.853938 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756b939-0f66-4267-8d15-a53257ec3893 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:09.854221 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756b939-0f66-4267-8d15-a53257ec3893 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:09.854504 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5756b939-0f66-4267-8d15-a53257ec3893 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:09.856322 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5756b939-0f66-4267-8d15-a53257ec3893 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:09.923633 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756b939-0f66-4267-8d15-a53257ec3893 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:09.925285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5756b939-0f66-4267-8d15-a53257ec3893 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:09.936865 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1213/2425] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:40:13.609376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e586b23-d1f3-40f6-9ce1-9f22b8e2b42b service neutron] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:13.611373 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-1e586b23-d1f3-40f6-9ce1-9f22b8e2b42b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:13.686210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e586b23-d1f3-40f6-9ce1-9f22b8e2b42b service neutron] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:13.692201 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f4d90-68f5-4ee8-a52e-543ff2d128ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46c99f43a9344d7d92632990e45bb9cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:13.692783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f4d90-68f5-4ee8-a52e-543ff2d128ab service placement] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:13.693072 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f4d90-68f5-4ee8-a52e-543ff2d128ab service placement] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:13.693354 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90f4d90-68f5-4ee8-a52e-543ff2d128ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:13.695124 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-e90f4d90-68f5-4ee8-a52e-543ff2d128ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:13.706608 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f4d90-68f5-4ee8-a52e-543ff2d128ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:13.708231 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90f4d90-68f5-4ee8-a52e-543ff2d128ab service placement] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:13.718240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1213/2426] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:40:13.813913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756b939-0f66-4267-8d15-a53257ec3893 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:13.816702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5756b939-0f66-4267-8d15-a53257ec3893 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:13.835231 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5756b939-0f66-4267-8d15-a53257ec3893 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:13.839899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ed82d-001e-4125-bd3d-3e7ccd2c01bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:13.840771 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ed82d-001e-4125-bd3d-3e7ccd2c01bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:13.841060 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ed82d-001e-4125-bd3d-3e7ccd2c01bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:13.841333 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ed82d-001e-4125-bd3d-3e7ccd2c01bb admin admin] PATH_INFO: `/v3/users/c7f8abef711c4cd2b53852b3a969b120` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:13.842520 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ed82d-001e-4125-bd3d-3e7ccd2c01bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7f8abef711c4cd2b53852b3a969b120)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:13.844499 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ed82d-001e-4125-bd3d-3e7ccd2c01bb admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:14.139008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d52ed82d-001e-4125-bd3d-3e7ccd2c01bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7f8abef711c4cd2b53852b3a969b120'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:14.139700 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d52ed82d-001e-4125-bd3d-3e7ccd2c01bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7f8abef711c4cd2b53852b3a969b120'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:14.156336 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d52ed82d-001e-4125-bd3d-3e7ccd2c01bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7f8abef711c4cd2b53852b3a969b120'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:14.157059 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d52ed82d-001e-4125-bd3d-3e7ccd2c01bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7f8abef711c4cd2b53852b3a969b120'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:14.295079 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-d52ed82d-001e-4125-bd3d-3e7ccd2c01bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7f8abef711c4cd2b53852b3a969b120'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:14.313778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1214/2427] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:40:13 2022] DELETE /identity/v3/users/c7f8abef711c4cd2b53852b3a969b120 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:14.741608 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f4d90-68f5-4ee8-a52e-543ff2d128ab service placement] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:14.754842 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90f4d90-68f5-4ee8-a52e-543ff2d128ab service placement] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:14.884987 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ff2152-6585-4847-89e8-86eb1e08ffb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:14.887344 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff2152-6585-4847-89e8-86eb1e08ffb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:14.888077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff2152-6585-4847-89e8-86eb1e08ffb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:14.890169 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ff2152-6585-4847-89e8-86eb1e08ffb8 admin admin] PATH_INFO: `/v3/projects/8244c746798443a1a155ae40853e5e26` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:14.893357 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff2152-6585-4847-89e8-86eb1e08ffb8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8244c746798443a1a155ae40853e5e26)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:14.896614 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ff2152-6585-4847-89e8-86eb1e08ffb8 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:15.054052 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:15.063650 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a5ff2152-6585-4847-89e8-86eb1e08ffb8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8244c746798443a1a155ae40853e5e26', 'request_id': 'req-a5ff2152-6585-4847-89e8-86eb1e08ffb8'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:15.071455 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a5ff2152-6585-4847-89e8-86eb1e08ffb8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8244c746798443a1a155ae40853e5e26', 'request_id': 'req-a5ff2152-6585-4847-89e8-86eb1e08ffb8'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:15.077544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a5ff2152-6585-4847-89e8-86eb1e08ffb8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8244c746798443a1a155ae40853e5e26', 'request_id': 'req-a5ff2152-6585-4847-89e8-86eb1e08ffb8'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:15.206347 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a5ff2152-6585-4847-89e8-86eb1e08ffb8 admin admin] The token cache is being invalidate because project 8244c746798443a1a155ae40853e5e26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:40:15.206748 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a5ff2152-6585-4847-89e8-86eb1e08ffb8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:15.207923 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-a5ff2152-6585-4847-89e8-86eb1e08ffb8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:15.211593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1214/2428] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:40:14 2022] DELETE /identity/v3/projects/8244c746798443a1a155ae40853e5e26 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:15.216893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2ea10-c008-4996-ad6e-e6a86fc10188 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:15.217380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2ea10-c008-4996-ad6e-e6a86fc10188 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:15.217501 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea2ea10-c008-4996-ad6e-e6a86fc10188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:15.248970 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9ea2ea10-c008-4996-ad6e-e6a86fc10188 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:15.250238 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9ea2ea10-c008-4996-ad6e-e6a86fc10188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:15.332761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:15.343935 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1215/2429] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:15.353898 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefda35-aa22-4919-a257-f37997f0ba82 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:15.354189 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefda35-aa22-4919-a257-f37997f0ba82 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:15.354470 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cefda35-aa22-4919-a257-f37997f0ba82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:15.377930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-6cefda35-aa22-4919-a257-f37997f0ba82 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:15.379177 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-6cefda35-aa22-4919-a257-f37997f0ba82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:15.388155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1215/2430] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:15.393007 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2ea10-c008-4996-ad6e-e6a86fc10188 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:15.403653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea2ea10-c008-4996-ad6e-e6a86fc10188 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:15.409644 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e8631-5682-455c-91ca-444f4811afb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:15.410575 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e8631-5682-455c-91ca-444f4811afb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:15.410859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e8631-5682-455c-91ca-444f4811afb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:15.411134 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1e8631-5682-455c-91ca-444f4811afb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:15.411917 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e8631-5682-455c-91ca-444f4811afb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:40:15.412223 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e8631-5682-455c-91ca-444f4811afb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:15.413836 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1e8631-5682-455c-91ca-444f4811afb3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:15.427171 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1216/2431] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:40:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:40:15.432514 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefda35-aa22-4919-a257-f37997f0ba82 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:15.443486 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cefda35-aa22-4919-a257-f37997f0ba82 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:15.448889 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3ca40-2418-43eb-b70b-f4155d4aee32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:15.449611 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3ca40-2418-43eb-b70b-f4155d4aee32 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:15.449963 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3ca40-2418-43eb-b70b-f4155d4aee32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:15.450300 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc3ca40-2418-43eb-b70b-f4155d4aee32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:15.451096 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc3ca40-2418-43eb-b70b-f4155d4aee32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:15.453205 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc3ca40-2418-43eb-b70b-f4155d4aee32 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:15.588222 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1216/2432] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:40:15 2022] POST /identity/v3/projects => generated 361 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:40:15.589267 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e8631-5682-455c-91ca-444f4811afb3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:15.599820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1e8631-5682-455c-91ca-444f4811afb3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:15.690875 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8eefa4-0893-47c4-bcd8-f7405ed81c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:15.691704 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8eefa4-0893-47c4-bcd8-f7405ed81c60 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:15.692410 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8eefa4-0893-47c4-bcd8-f7405ed81c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:15.693053 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8eefa4-0893-47c4-bcd8-f7405ed81c60 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:15.694219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8eefa4-0893-47c4-bcd8-f7405ed81c60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:15.696604 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8eefa4-0893-47c4-bcd8-f7405ed81c60 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:15.800376 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1217/2433] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:40:15 2022] POST /identity/v3/users => generated 281 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:40:15.804185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3ca40-2418-43eb-b70b-f4155d4aee32 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:15.825470 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc3ca40-2418-43eb-b70b-f4155d4aee32 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:15.831362 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b494686-b602-4715-8aa3-abfd0883c88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:15.832330 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b494686-b602-4715-8aa3-abfd0883c88a admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:15.832756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b494686-b602-4715-8aa3-abfd0883c88a admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:15.833180 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b494686-b602-4715-8aa3-abfd0883c88a admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:15.834076 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b494686-b602-4715-8aa3-abfd0883c88a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:15.836614 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b494686-b602-4715-8aa3-abfd0883c88a admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:15.849648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1217/2434] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:40:15.853880 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8eefa4-0893-47c4-bcd8-f7405ed81c60 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:15.868260 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8eefa4-0893-47c4-bcd8-f7405ed81c60 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:15.877276 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9506a2d4-0d28-4621-bce6-66d38c3db4d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:15.877276 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506a2d4-0d28-4621-bce6-66d38c3db4d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:15.877276 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506a2d4-0d28-4621-bce6-66d38c3db4d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:15.877276 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9506a2d4-0d28-4621-bce6-66d38c3db4d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:15.877685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9506a2d4-0d28-4621-bce6-66d38c3db4d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:15.879950 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9506a2d4-0d28-4621-bce6-66d38c3db4d8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:15.894506 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1218/2435] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:15 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:40:15.897026 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b494686-b602-4715-8aa3-abfd0883c88a admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:15.912193 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b494686-b602-4715-8aa3-abfd0883c88a admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:15.917720 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178162e-786a-4899-b5b4-13fbf5b13219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:15.918888 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178162e-786a-4899-b5b4-13fbf5b13219 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:15.919933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178162e-786a-4899-b5b4-13fbf5b13219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:15.921331 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178162e-786a-4899-b5b4-13fbf5b13219 admin admin] PATH_INFO: `/v3/projects/7ecae66267aa46c58878219da5a3a00b/users/fadc0de1301b4418a7a78a3f63e1ad2b/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:15.947105 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178162e-786a-4899-b5b4-13fbf5b13219 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ecae66267aa46c58878219da5a3a00b, user_id=fadc0de1301b4418a7a78a3f63e1ad2b, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:15.950338 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178162e-786a-4899-b5b4-13fbf5b13219 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:16.051602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1218/2436] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:40:15 2022] PUT /identity/v3/projects/7ecae66267aa46c58878219da5a3a00b/users/fadc0de1301b4418a7a78a3f63e1ad2b/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:16.062111 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6fdb8b-b331-455e-9a34-af1a74fcfe21 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:16.062403 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6fdb8b-b331-455e-9a34-af1a74fcfe21 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:16.062768 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6fdb8b-b331-455e-9a34-af1a74fcfe21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:16.116571 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-bc6fdb8b-b331-455e-9a34-af1a74fcfe21 None None] MFA Rules not processed for user `fadc0de1301b4418a7a78a3f63e1ad2b`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:16.116571 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-bc6fdb8b-b331-455e-9a34-af1a74fcfe21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:16.199248 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1219/2437] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:16.203570 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178162e-786a-4899-b5b4-13fbf5b13219 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:16.272602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178162e-786a-4899-b5b4-13fbf5b13219 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:16.277808 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159aaf7-9656-4eed-a7fb-40a16a1c8894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:16.278481 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159aaf7-9656-4eed-a7fb-40a16a1c8894 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:16.278773 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159aaf7-9656-4eed-a7fb-40a16a1c8894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:16.279053 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3159aaf7-9656-4eed-a7fb-40a16a1c8894 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:16.279801 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3159aaf7-9656-4eed-a7fb-40a16a1c8894 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:16.281729 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3159aaf7-9656-4eed-a7fb-40a16a1c8894 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:16.384981 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1219/2438] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:40:16 2022] POST /identity/v3/projects => generated 363 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:40:16.387161 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6fdb8b-b331-455e-9a34-af1a74fcfe21 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:16.399142 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6fdb8b-b331-455e-9a34-af1a74fcfe21 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:16.473785 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6003dcc-44b8-4f70-b2b1-c7c69c999c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:16.474550 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6003dcc-44b8-4f70-b2b1-c7c69c999c30 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:16.476653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6003dcc-44b8-4f70-b2b1-c7c69c999c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:16.476653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6003dcc-44b8-4f70-b2b1-c7c69c999c30 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:16.476653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6003dcc-44b8-4f70-b2b1-c7c69c999c30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:16.477860 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6003dcc-44b8-4f70-b2b1-c7c69c999c30 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:16.579182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1220/2439] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:40:16 2022] POST /identity/v3/users => generated 288 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:40:16.583993 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159aaf7-9656-4eed-a7fb-40a16a1c8894 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:16.595328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3159aaf7-9656-4eed-a7fb-40a16a1c8894 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:16.608076 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20972ec-be44-4c59-a12f-c060ad4bcc57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:16.609087 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20972ec-be44-4c59-a12f-c060ad4bcc57 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:16.609620 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20972ec-be44-4c59-a12f-c060ad4bcc57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:16.610271 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20972ec-be44-4c59-a12f-c060ad4bcc57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:16.611340 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20972ec-be44-4c59-a12f-c060ad4bcc57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:16.615682 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20972ec-be44-4c59-a12f-c060ad4bcc57 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:16.642225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1220/2440] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:40:16.644336 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6003dcc-44b8-4f70-b2b1-c7c69c999c30 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:16.655827 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6003dcc-44b8-4f70-b2b1-c7c69c999c30 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:16.662495 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8847ea-79c8-4f15-a49a-ebf7af33e074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:16.664073 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8847ea-79c8-4f15-a49a-ebf7af33e074 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:16.664572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8847ea-79c8-4f15-a49a-ebf7af33e074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:16.665069 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8847ea-79c8-4f15-a49a-ebf7af33e074 admin admin] PATH_INFO: `/v3/projects/539418d91c5346138774d34aedab70c3/users/a2c70ed851d744e9872c64e21fc3c052/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:16.684263 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8847ea-79c8-4f15-a49a-ebf7af33e074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=539418d91c5346138774d34aedab70c3, user_id=a2c70ed851d744e9872c64e21fc3c052, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:16.687474 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8847ea-79c8-4f15-a49a-ebf7af33e074 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:16.784186 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1221/2441] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:40:16 2022] PUT /identity/v3/projects/539418d91c5346138774d34aedab70c3/users/a2c70ed851d744e9872c64e21fc3c052/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:16.785913 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20972ec-be44-4c59-a12f-c060ad4bcc57 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:16.796184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20972ec-be44-4c59-a12f-c060ad4bcc57 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:16.827322 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:16.832208 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:16.855563 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9609ea44-d473-49b5-b411-4607193fd0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:16.856237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609ea44-d473-49b5-b411-4607193fd0f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:16.856523 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609ea44-d473-49b5-b411-4607193fd0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:16.856796 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9609ea44-d473-49b5-b411-4607193fd0f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:16.857543 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609ea44-d473-49b5-b411-4607193fd0f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:16.859173 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9609ea44-d473-49b5-b411-4607193fd0f9 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:16.873064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1221/2442] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:16 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:40:16.879922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8847ea-79c8-4f15-a49a-ebf7af33e074 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:16.885882 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8847ea-79c8-4f15-a49a-ebf7af33e074 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:16.890724 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b8382-b8ec-4546-88bd-ddbb268e4357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:16.891730 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b8382-b8ec-4546-88bd-ddbb268e4357 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:16.891955 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b8382-b8ec-4546-88bd-ddbb268e4357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:16.892235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b8382-b8ec-4546-88bd-ddbb268e4357 admin admin] PATH_INFO: `/v3/projects/539418d91c5346138774d34aedab70c3/users/a2c70ed851d744e9872c64e21fc3c052/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:16.894634 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b8382-b8ec-4546-88bd-ddbb268e4357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=539418d91c5346138774d34aedab70c3, user_id=a2c70ed851d744e9872c64e21fc3c052, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:16.897366 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b8382-b8ec-4546-88bd-ddbb268e4357 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:16.908874 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1222/2443] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:40:16 2022] PUT /identity/v3/projects/539418d91c5346138774d34aedab70c3/users/a2c70ed851d744e9872c64e21fc3c052/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:16.919996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53e358-a213-43e7-85de-b52ac9015c86 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:16.920287 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53e358-a213-43e7-85de-b52ac9015c86 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:16.920565 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53e358-a213-43e7-85de-b52ac9015c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:16.951775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-0b53e358-a213-43e7-85de-b52ac9015c86 None None] MFA Rules not processed for user `a2c70ed851d744e9872c64e21fc3c052`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:16.953491 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-0b53e358-a213-43e7-85de-b52ac9015c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:17.038621 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1222/2444] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:16 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:17.045869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b8382-b8ec-4546-88bd-ddbb268e4357 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:17.047612 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3c1b8382-b8ec-4546-88bd-ddbb268e4357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:17.120769 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b8382-b8ec-4546-88bd-ddbb268e4357 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:17.133385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa4e77-f25c-4ad4-83c2-0dff6b68d048 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:17.134392 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa4e77-f25c-4ad4-83c2-0dff6b68d048 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:17.134772 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa4e77-f25c-4ad4-83c2-0dff6b68d048 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:17.135160 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa4e77-f25c-4ad4-83c2-0dff6b68d048 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:17.155794 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa4e77-f25c-4ad4-83c2-0dff6b68d048 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:17.169518 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fa4e77-f25c-4ad4-83c2-0dff6b68d048 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:17.170643 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1223/2445] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:17 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:40:17.484330 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53e358-a213-43e7-85de-b52ac9015c86 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:17.498902 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53e358-a213-43e7-85de-b52ac9015c86 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:17.503347 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42023a75-b975-4f52-93ed-83db5f4e3728 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:17.504242 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42023a75-b975-4f52-93ed-83db5f4e3728 service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:17.504531 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42023a75-b975-4f52-93ed-83db5f4e3728 service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:17.504809 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42023a75-b975-4f52-93ed-83db5f4e3728 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:17.590447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42023a75-b975-4f52-93ed-83db5f4e3728 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:17.592097 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42023a75-b975-4f52-93ed-83db5f4e3728 service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:17.603687 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1223/2446] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:17 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:40:23.470265 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea8ded-a9b2-4148-9748-70b4c163b38e None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:23.471358 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea8ded-a9b2-4148-9748-70b4c163b38e None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:23.471358 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ea8ded-a9b2-4148-9748-70b4c163b38e None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:23.475336 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1224/2447] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jun 30 10:40:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:40:23.480826 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862d02c-1711-4a6b-b126-3b3bdd44c73a None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:23.481350 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862d02c-1711-4a6b-b126-3b3bdd44c73a None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:23.481537 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862d02c-1711-4a6b-b126-3b3bdd44c73a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:23.507246 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-5862d02c-1711-4a6b-b126-3b3bdd44c73a None None] MFA Rules not processed for user `059b05007c0144f7a50dbd9f0ef41359`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:23.508649 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-5862d02c-1711-4a6b-b126-3b3bdd44c73a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:23.566020 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1224/2448] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jun 30 10:40:23 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:23.578071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea8ded-a9b2-4148-9748-70b4c163b38e None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:23.580393 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-37ea8ded-a9b2-4148-9748-70b4c163b38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:23.593461 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ea8ded-a9b2-4148-9748-70b4c163b38e None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:23.597832 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de300d9-843c-4250-a962-501a176cb6c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:23.598476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de300d9-843c-4250-a962-501a176cb6c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:23.598765 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de300d9-843c-4250-a962-501a176cb6c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:23.599048 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de300d9-843c-4250-a962-501a176cb6c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:23.606287 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de300d9-843c-4250-a962-501a176cb6c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:23.607763 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de300d9-843c-4250-a962-501a176cb6c3 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:23.617619 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1225/2449] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:23 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:40:23.711959 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2a3ca-1c36-4291-b66e-4848d5cbc1e5 None None] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:23.712255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2a3ca-1c36-4291-b66e-4848d5cbc1e5 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:23.712540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b2a3ca-1c36-4291-b66e-4848d5cbc1e5 None None] PATH_INFO: `/` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:23.713915 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1225/2450] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jun 30 10:40:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 30 10:40:23.719666 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768a57e-08ac-4862-ae4c-0a7156cedca9 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:23.719957 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768a57e-08ac-4862-ae4c-0a7156cedca9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:23.720258 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768a57e-08ac-4862-ae4c-0a7156cedca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:23.758531 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7768a57e-08ac-4862-ae4c-0a7156cedca9 None None] MFA Rules not processed for user `71f6d7715e6f49ffb9fdf6e1307bbd51`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:23.759809 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7768a57e-08ac-4862-ae4c-0a7156cedca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:23.827342 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1226/2451] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jun 30 10:40:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:23.838785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2a3ca-1c36-4291-b66e-4848d5cbc1e5 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:23.840536 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-75b2a3ca-1c36-4291-b66e-4848d5cbc1e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:23.848480 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:23.854438 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b2a3ca-1c36-4291-b66e-4848d5cbc1e5 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:23.857999 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370da5d-8330-463c-8764-2056aa8f2f3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:23.858644 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370da5d-8330-463c-8764-2056aa8f2f3a service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:23.858971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370da5d-8330-463c-8764-2056aa8f2f3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:23.859252 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370da5d-8330-463c-8764-2056aa8f2f3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:23.869405 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370da5d-8330-463c-8764-2056aa8f2f3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:23.870842 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370da5d-8330-463c-8764-2056aa8f2f3a service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:23.882124 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1226/2452] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:40:26.117372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768a57e-08ac-4862-ae4c-0a7156cedca9 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:26.128170 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768a57e-08ac-4862-ae4c-0a7156cedca9 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:26.131715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584c6ac-07d2-448d-97e5-8a26c9cd3631 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:26.132355 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584c6ac-07d2-448d-97e5-8a26c9cd3631 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:26.132638 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584c6ac-07d2-448d-97e5-8a26c9cd3631 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:26.132912 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b584c6ac-07d2-448d-97e5-8a26c9cd3631 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:26.134930 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b584c6ac-07d2-448d-97e5-8a26c9cd3631 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:26.148100 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:26.198062 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584c6ac-07d2-448d-97e5-8a26c9cd3631 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:26.199672 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b584c6ac-07d2-448d-97e5-8a26c9cd3631 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:26.209880 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1227/2453] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:40:30.635105 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370da5d-8330-463c-8764-2056aa8f2f3a service neutron] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:30.637392 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-8370da5d-8330-463c-8764-2056aa8f2f3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:30.655590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370da5d-8330-463c-8764-2056aa8f2f3a service neutron] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:30.662306 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df647c-e633-4bc6-963a-d42bc3e0a394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:30.663148 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df647c-e633-4bc6-963a-d42bc3e0a394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:30.663426 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df647c-e633-4bc6-963a-d42bc3e0a394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:30.663752 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88df647c-e633-4bc6-963a-d42bc3e0a394 admin admin] PATH_INFO: `/v3/users/797a7b1e128f4102bac6430a60d2c0dc` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:30.664926 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df647c-e633-4bc6-963a-d42bc3e0a394 admin admin] RBAC: Authorizing `identity:delete_user(user_id=797a7b1e128f4102bac6430a60d2c0dc)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:30.666869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88df647c-e633-4bc6-963a-d42bc3e0a394 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:30.933785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-88df647c-e633-4bc6-963a-d42bc3e0a394 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '797a7b1e128f4102bac6430a60d2c0dc'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:31.071762 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-88df647c-e633-4bc6-963a-d42bc3e0a394 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '797a7b1e128f4102bac6430a60d2c0dc'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:31.085252 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-88df647c-e633-4bc6-963a-d42bc3e0a394 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '797a7b1e128f4102bac6430a60d2c0dc'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:31.087214 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-88df647c-e633-4bc6-963a-d42bc3e0a394 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '797a7b1e128f4102bac6430a60d2c0dc'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:31.087214 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-88df647c-e633-4bc6-963a-d42bc3e0a394 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '797a7b1e128f4102bac6430a60d2c0dc'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:31.099437 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1227/2454] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:40:30 2022] DELETE /identity/v3/users/797a7b1e128f4102bac6430a60d2c0dc => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:31.414033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584c6ac-07d2-448d-97e5-8a26c9cd3631 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:31.424818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b584c6ac-07d2-448d-97e5-8a26c9cd3631 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:31.507502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb40a911-4bfd-4979-86fe-0619273e157d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:31.508315 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40a911-4bfd-4979-86fe-0619273e157d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:31.508602 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40a911-4bfd-4979-86fe-0619273e157d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:31.508883 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40a911-4bfd-4979-86fe-0619273e157d admin admin] PATH_INFO: `/v3/projects/c8ee4130595948df9d462a6ca9f49c10` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:31.510149 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb40a911-4bfd-4979-86fe-0619273e157d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8ee4130595948df9d462a6ca9f49c10)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:31.512162 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb40a911-4bfd-4979-86fe-0619273e157d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:31.708801 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-eb40a911-4bfd-4979-86fe-0619273e157d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8ee4130595948df9d462a6ca9f49c10', 'request_id': 'req-eb40a911-4bfd-4979-86fe-0619273e157d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:31.786995 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df647c-e633-4bc6-963a-d42bc3e0a394 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:31.788633 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-88df647c-e633-4bc6-963a-d42bc3e0a394 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:31.798833 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-eb40a911-4bfd-4979-86fe-0619273e157d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8ee4130595948df9d462a6ca9f49c10', 'request_id': 'req-eb40a911-4bfd-4979-86fe-0619273e157d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:31.802943 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88df647c-e633-4bc6-963a-d42bc3e0a394 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:31.813129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-eb40a911-4bfd-4979-86fe-0619273e157d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8ee4130595948df9d462a6ca9f49c10', 'request_id': 'req-eb40a911-4bfd-4979-86fe-0619273e157d'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:31.828000 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-eb40a911-4bfd-4979-86fe-0619273e157d admin admin] The token cache is being invalidate because project c8ee4130595948df9d462a6ca9f49c10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:40:31.828296 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-eb40a911-4bfd-4979-86fe-0619273e157d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:31.829414 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-eb40a911-4bfd-4979-86fe-0619273e157d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:31.831128 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1228/2455] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:40:31 2022] DELETE /identity/v3/projects/c8ee4130595948df9d462a6ca9f49c10 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:31.836110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285df36-d148-4d69-b8db-d1e32e355443 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:31.836379 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285df36-d148-4d69-b8db-d1e32e355443 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:31.836660 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f285df36-d148-4d69-b8db-d1e32e355443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:31.858365 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f285df36-d148-4d69-b8db-d1e32e355443 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:31.861582 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f285df36-d148-4d69-b8db-d1e32e355443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:31.870142 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172971e-5133-4c91-95b2-9a06cbccbef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:31.871038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172971e-5133-4c91-95b2-9a06cbccbef3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:31.871348 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172971e-5133-4c91-95b2-9a06cbccbef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:31.871835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172971e-5133-4c91-95b2-9a06cbccbef3 admin admin] PATH_INFO: `/v3/users/fadc0de1301b4418a7a78a3f63e1ad2b` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:31.873357 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e172971e-5133-4c91-95b2-9a06cbccbef3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fadc0de1301b4418a7a78a3f63e1ad2b)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:31.875299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e172971e-5133-4c91-95b2-9a06cbccbef3 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:31.904001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:31.931185 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1229/2456] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:40:31.939361 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a8eca-ca07-4845-8cbe-f8aff9dccf07 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:31.939657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a8eca-ca07-4845-8cbe-f8aff9dccf07 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:31.939961 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a8eca-ca07-4845-8cbe-f8aff9dccf07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:31.967698 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e10a8eca-ca07-4845-8cbe-f8aff9dccf07 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:31.968046 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e10a8eca-ca07-4845-8cbe-f8aff9dccf07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:31.977093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1230/2457] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:40:31.981442 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a8eca-ca07-4845-8cbe-f8aff9dccf07 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:31.992971 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a8eca-ca07-4845-8cbe-f8aff9dccf07 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:31.997783 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a13bf-6755-46ac-b227-637d1aeaf31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:31.998564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a13bf-6755-46ac-b227-637d1aeaf31a admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:31.998851 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a13bf-6755-46ac-b227-637d1aeaf31a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:31.999133 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50a13bf-6755-46ac-b227-637d1aeaf31a admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:31.999900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a13bf-6755-46ac-b227-637d1aeaf31a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:40:32.000201 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a13bf-6755-46ac-b227-637d1aeaf31a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:32.002541 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50a13bf-6755-46ac-b227-637d1aeaf31a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:32.022182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1231/2458] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:40:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:40:32.023224 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a13bf-6755-46ac-b227-637d1aeaf31a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:32.033540 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50a13bf-6755-46ac-b227-637d1aeaf31a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:32.038646 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447fdb0-c9a9-492e-97df-eb8447b22d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:32.039374 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447fdb0-c9a9-492e-97df-eb8447b22d28 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:32.039685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447fdb0-c9a9-492e-97df-eb8447b22d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:32.039958 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447fdb0-c9a9-492e-97df-eb8447b22d28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:32.040645 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447fdb0-c9a9-492e-97df-eb8447b22d28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:32.042792 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447fdb0-c9a9-492e-97df-eb8447b22d28 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:32.169240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e172971e-5133-4c91-95b2-9a06cbccbef3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fadc0de1301b4418a7a78a3f63e1ad2b'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:32.225166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1232/2459] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:40:32 2022] POST /identity/v3/projects => generated 383 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:40:32.229332 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447fdb0-c9a9-492e-97df-eb8447b22d28 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:32.283465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e172971e-5133-4c91-95b2-9a06cbccbef3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fadc0de1301b4418a7a78a3f63e1ad2b'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:32.291819 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:32.292400 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447fdb0-c9a9-492e-97df-eb8447b22d28 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:32.297423 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b20a49-0239-4611-a2c9-68cd7d9def80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:32.298293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b20a49-0239-4611-a2c9-68cd7d9def80 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:32.298804 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b20a49-0239-4611-a2c9-68cd7d9def80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:32.299092 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b20a49-0239-4611-a2c9-68cd7d9def80 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:32.299841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b20a49-0239-4611-a2c9-68cd7d9def80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:32.301485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e172971e-5133-4c91-95b2-9a06cbccbef3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fadc0de1301b4418a7a78a3f63e1ad2b'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:32.302282 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e172971e-5133-4c91-95b2-9a06cbccbef3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fadc0de1301b4418a7a78a3f63e1ad2b'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:32.302397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b20a49-0239-4611-a2c9-68cd7d9def80 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:32.302900 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e172971e-5133-4c91-95b2-9a06cbccbef3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fadc0de1301b4418a7a78a3f63e1ad2b'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:32.315636 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1228/2460] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:40:31 2022] DELETE /identity/v3/users/fadc0de1301b4418a7a78a3f63e1ad2b => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:32.316544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172971e-5133-4c91-95b2-9a06cbccbef3 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:32.317883 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-e172971e-5133-4c91-95b2-9a06cbccbef3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:32.369523 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1233/2461] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:40:32 2022] POST /identity/v3/users => generated 292 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:40:32.373930 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510895d2-80d2-4f85-acef-e36bd32780c2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:32.374226 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510895d2-80d2-4f85-acef-e36bd32780c2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:32.374511 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510895d2-80d2-4f85-acef-e36bd32780c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:32.377698 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e172971e-5133-4c91-95b2-9a06cbccbef3 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:32.381350 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224488df-1393-40f7-adbe-b3d9a34dbb8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:32.382002 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224488df-1393-40f7-adbe-b3d9a34dbb8c service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:32.382282 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224488df-1393-40f7-adbe-b3d9a34dbb8c service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:32.382555 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224488df-1393-40f7-adbe-b3d9a34dbb8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:32.384390 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-224488df-1393-40f7-adbe-b3d9a34dbb8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:32.392631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-510895d2-80d2-4f85-acef-e36bd32780c2 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:32.393839 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-510895d2-80d2-4f85-acef-e36bd32780c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:32.449844 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224488df-1393-40f7-adbe-b3d9a34dbb8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:32.451509 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224488df-1393-40f7-adbe-b3d9a34dbb8c service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:32.458495 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1234/2462] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:32.458495 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510895d2-80d2-4f85-acef-e36bd32780c2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:32.458495 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-510895d2-80d2-4f85-acef-e36bd32780c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:32.463036 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1229/2463] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:32 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:40:32.466890 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224488df-1393-40f7-adbe-b3d9a34dbb8c service neutron] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:32.469551 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510895d2-80d2-4f85-acef-e36bd32780c2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:32.475620 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661496a-be31-4cff-911d-b2dd4d7d1d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:32.476218 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224488df-1393-40f7-adbe-b3d9a34dbb8c service neutron] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:32.476353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661496a-be31-4cff-911d-b2dd4d7d1d37 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:32.476636 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661496a-be31-4cff-911d-b2dd4d7d1d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:32.479000 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661496a-be31-4cff-911d-b2dd4d7d1d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:32.479000 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c661496a-be31-4cff-911d-b2dd4d7d1d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:32.479454 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c661496a-be31-4cff-911d-b2dd4d7d1d37 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:32.482229 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd21e0-d9ec-4b77-a7da-3dd2987ae32a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:32.482886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd21e0-d9ec-4b77-a7da-3dd2987ae32a service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:32.483173 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd21e0-d9ec-4b77-a7da-3dd2987ae32a service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:32.483487 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd21e0-d9ec-4b77-a7da-3dd2987ae32a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:32.493835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd21e0-d9ec-4b77-a7da-3dd2987ae32a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:32.495021 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1235/2464] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:40:32.496082 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fd21e0-d9ec-4b77-a7da-3dd2987ae32a service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:32.499405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661496a-be31-4cff-911d-b2dd4d7d1d37 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:32.508076 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1230/2465] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:40:32.516407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661496a-be31-4cff-911d-b2dd4d7d1d37 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:32.521755 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96fc44-7dc4-49a1-8dca-e76d92cb748c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:32.522578 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96fc44-7dc4-49a1-8dca-e76d92cb748c admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:32.522710 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96fc44-7dc4-49a1-8dca-e76d92cb748c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:32.522985 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96fc44-7dc4-49a1-8dca-e76d92cb748c admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:32.523764 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b96fc44-7dc4-49a1-8dca-e76d92cb748c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:32.525389 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b96fc44-7dc4-49a1-8dca-e76d92cb748c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:32.538882 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1236/2466] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:32 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:40:32.541025 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd21e0-d9ec-4b77-a7da-3dd2987ae32a service neutron] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:32.551692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd21e0-d9ec-4b77-a7da-3dd2987ae32a service neutron] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:32.556803 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5424a-8e58-4f70-b795-8a9ada73717e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:32.557697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5424a-8e58-4f70-b795-8a9ada73717e admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:32.557986 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5424a-8e58-4f70-b795-8a9ada73717e admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:32.558270 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c5424a-8e58-4f70-b795-8a9ada73717e admin admin] PATH_INFO: `/v3/projects/80ba2ca60b7e4578b9334dd6eb013eec/users/fa1c093874a74439ba62aa051946e52a/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:32.574089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5424a-8e58-4f70-b795-8a9ada73717e admin admin] RBAC: Authorizing `identity:create_grant(project_id=80ba2ca60b7e4578b9334dd6eb013eec, user_id=fa1c093874a74439ba62aa051946e52a, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:32.576808 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c5424a-8e58-4f70-b795-8a9ada73717e admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:32.687571 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1231/2467] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:40:32 2022] PUT /identity/v3/projects/80ba2ca60b7e4578b9334dd6eb013eec/users/fa1c093874a74439ba62aa051946e52a/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:32.699027 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd6371-4fd4-40f5-bfcc-4126d2f8f08b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:32.699437 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd6371-4fd4-40f5-bfcc-4126d2f8f08b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:32.699808 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fd6371-4fd4-40f5-bfcc-4126d2f8f08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:32.714472 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:32.732141 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-08fd6371-4fd4-40f5-bfcc-4126d2f8f08b None None] MFA Rules not processed for user `fa1c093874a74439ba62aa051946e52a`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:32.733277 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-08fd6371-4fd4-40f5-bfcc-4126d2f8f08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:32.816560 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1237/2468] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:32.830804 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5424a-8e58-4f70-b795-8a9ada73717e admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:32.832669 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-23c5424a-8e58-4f70-b795-8a9ada73717e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:32.894758 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=98,keystone:DELETE=1,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:32.919677 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c5424a-8e58-4f70-b795-8a9ada73717e admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:32.925824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b358e5-4f5f-43cc-9304-b9344b9b8c8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:32.926539 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b358e5-4f5f-43cc-9304-b9344b9b8c8f service nova] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:32.926826 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b358e5-4f5f-43cc-9304-b9344b9b8c8f service nova] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:32.927107 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b358e5-4f5f-43cc-9304-b9344b9b8c8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:32.938136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b358e5-4f5f-43cc-9304-b9344b9b8c8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:32.941023 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b358e5-4f5f-43cc-9304-b9344b9b8c8f service nova] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:32.957843 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1232/2469] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:40:33.104869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd6371-4fd4-40f5-bfcc-4126d2f8f08b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:33.106578 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-08fd6371-4fd4-40f5-bfcc-4126d2f8f08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:33.180792 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fd6371-4fd4-40f5-bfcc-4126d2f8f08b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:33.185385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda27baa-aeef-4e65-af77-d215a1b3c508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:33.186277 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda27baa-aeef-4e65-af77-d215a1b3c508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:33.186557 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda27baa-aeef-4e65-af77-d215a1b3c508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:33.186829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda27baa-aeef-4e65-af77-d215a1b3c508 admin admin] PATH_INFO: `/v3/projects/7ecae66267aa46c58878219da5a3a00b` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:33.188040 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda27baa-aeef-4e65-af77-d215a1b3c508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ecae66267aa46c58878219da5a3a00b)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:33.191077 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda27baa-aeef-4e65-af77-d215a1b3c508 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:33.320955 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fda27baa-aeef-4e65-af77-d215a1b3c508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ecae66267aa46c58878219da5a3a00b', 'request_id': 'req-fda27baa-aeef-4e65-af77-d215a1b3c508'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:33.377746 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fda27baa-aeef-4e65-af77-d215a1b3c508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ecae66267aa46c58878219da5a3a00b', 'request_id': 'req-fda27baa-aeef-4e65-af77-d215a1b3c508'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:33.383939 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fda27baa-aeef-4e65-af77-d215a1b3c508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ecae66267aa46c58878219da5a3a00b', 'request_id': 'req-fda27baa-aeef-4e65-af77-d215a1b3c508'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:33.399672 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fda27baa-aeef-4e65-af77-d215a1b3c508 admin admin] The token cache is being invalidate because project 7ecae66267aa46c58878219da5a3a00b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:40:33.400168 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fda27baa-aeef-4e65-af77-d215a1b3c508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:33.401162 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fda27baa-aeef-4e65-af77-d215a1b3c508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:33.403144 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1238/2470] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:40:33 2022] DELETE /identity/v3/projects/7ecae66267aa46c58878219da5a3a00b => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:33.408714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b358e5-4f5f-43cc-9304-b9344b9b8c8f service nova] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:33.419679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b358e5-4f5f-43cc-9304-b9344b9b8c8f service nova] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:33.483143 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31136fbc-4a8c-4ad1-9835-c5b6ddc9ac60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:33.484086 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31136fbc-4a8c-4ad1-9835-c5b6ddc9ac60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:33.484374 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31136fbc-4a8c-4ad1-9835-c5b6ddc9ac60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:33.484656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31136fbc-4a8c-4ad1-9835-c5b6ddc9ac60 admin admin] PATH_INFO: `/v3/users/a2c70ed851d744e9872c64e21fc3c052` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:33.486672 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31136fbc-4a8c-4ad1-9835-c5b6ddc9ac60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2c70ed851d744e9872c64e21fc3c052)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:33.488673 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31136fbc-4a8c-4ad1-9835-c5b6ddc9ac60 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:33.733973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-31136fbc-4a8c-4ad1-9835-c5b6ddc9ac60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2c70ed851d744e9872c64e21fc3c052'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:33.821448 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-31136fbc-4a8c-4ad1-9835-c5b6ddc9ac60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2c70ed851d744e9872c64e21fc3c052'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:33.835625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-31136fbc-4a8c-4ad1-9835-c5b6ddc9ac60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2c70ed851d744e9872c64e21fc3c052'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:33.841700 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-31136fbc-4a8c-4ad1-9835-c5b6ddc9ac60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2c70ed851d744e9872c64e21fc3c052'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:33.841700 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-31136fbc-4a8c-4ad1-9835-c5b6ddc9ac60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2c70ed851d744e9872c64e21fc3c052'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:33.853807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1233/2471] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:40:33 2022] DELETE /identity/v3/users/a2c70ed851d744e9872c64e21fc3c052 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:34.413385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda27baa-aeef-4e65-af77-d215a1b3c508 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:34.415836 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fda27baa-aeef-4e65-af77-d215a1b3c508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:34.432068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda27baa-aeef-4e65-af77-d215a1b3c508 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:34.465976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:34.524815 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0074db9-9c30-4cec-8c0c-42428f8e7183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:34.524815 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0074db9-9c30-4cec-8c0c-42428f8e7183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:34.524815 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0074db9-9c30-4cec-8c0c-42428f8e7183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:34.524815 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0074db9-9c30-4cec-8c0c-42428f8e7183 admin admin] PATH_INFO: `/v3/projects/539418d91c5346138774d34aedab70c3` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:34.525485 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0074db9-9c30-4cec-8c0c-42428f8e7183 admin admin] RBAC: Authorizing `identity:delete_project(project_id=539418d91c5346138774d34aedab70c3)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:34.525485 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0074db9-9c30-4cec-8c0c-42428f8e7183 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:34.769726 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c0074db9-9c30-4cec-8c0c-42428f8e7183 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '539418d91c5346138774d34aedab70c3', 'request_id': 'req-c0074db9-9c30-4cec-8c0c-42428f8e7183'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:34.888941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c0074db9-9c30-4cec-8c0c-42428f8e7183 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '539418d91c5346138774d34aedab70c3', 'request_id': 'req-c0074db9-9c30-4cec-8c0c-42428f8e7183'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:34.896938 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c0074db9-9c30-4cec-8c0c-42428f8e7183 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '539418d91c5346138774d34aedab70c3', 'request_id': 'req-c0074db9-9c30-4cec-8c0c-42428f8e7183'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:34.909160 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c0074db9-9c30-4cec-8c0c-42428f8e7183 admin admin] The token cache is being invalidate because project 539418d91c5346138774d34aedab70c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:40:34.909536 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c0074db9-9c30-4cec-8c0c-42428f8e7183 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:34.910240 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c0074db9-9c30-4cec-8c0c-42428f8e7183 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:34.912109 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1239/2472] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:40:34 2022] DELETE /identity/v3/projects/539418d91c5346138774d34aedab70c3 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:34.916921 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01972f9d-9347-4adf-a14c-78592df6f2a2 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:34.917380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01972f9d-9347-4adf-a14c-78592df6f2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:34.917671 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01972f9d-9347-4adf-a14c-78592df6f2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:34.950601 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-01972f9d-9347-4adf-a14c-78592df6f2a2 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:34.956071 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-01972f9d-9347-4adf-a14c-78592df6f2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:35.005053 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:35.018816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1234/2473] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:35.026468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b0363-0c76-42d4-bcd7-55e772c2f9e0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:35.026765 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b0363-0c76-42d4-bcd7-55e772c2f9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:35.027051 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489b0363-0c76-42d4-bcd7-55e772c2f9e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:35.047103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-489b0363-0c76-42d4-bcd7-55e772c2f9e0 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:35.049198 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-489b0363-0c76-42d4-bcd7-55e772c2f9e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:35.059242 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1240/2474] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:35.065358 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01972f9d-9347-4adf-a14c-78592df6f2a2 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:35.075704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01972f9d-9347-4adf-a14c-78592df6f2a2 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:35.081234 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357fa0f0-d10a-4c6b-a649-2df4bf01ba70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:35.082096 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fa0f0-d10a-4c6b-a649-2df4bf01ba70 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:35.082554 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fa0f0-d10a-4c6b-a649-2df4bf01ba70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:35.082834 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357fa0f0-d10a-4c6b-a649-2df4bf01ba70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:35.083958 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357fa0f0-d10a-4c6b-a649-2df4bf01ba70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:40:35.084336 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357fa0f0-d10a-4c6b-a649-2df4bf01ba70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:35.086065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357fa0f0-d10a-4c6b-a649-2df4bf01ba70 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:35.098560 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1235/2475] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:40:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:40:35.111782 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b0363-0c76-42d4-bcd7-55e772c2f9e0 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:35.120862 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489b0363-0c76-42d4-bcd7-55e772c2f9e0 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:35.125562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fffd5-22e9-4e92-b800-e0ec85e29c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:35.126399 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fffd5-22e9-4e92-b800-e0ec85e29c2b admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:35.126895 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fffd5-22e9-4e92-b800-e0ec85e29c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:35.127060 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fffd5-22e9-4e92-b800-e0ec85e29c2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:35.127803 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11fffd5-22e9-4e92-b800-e0ec85e29c2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:35.129726 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11fffd5-22e9-4e92-b800-e0ec85e29c2b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:35.299143 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1241/2476] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:40:35 2022] POST /identity/v3/projects => generated 387 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:40:35.303396 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357fa0f0-d10a-4c6b-a649-2df4bf01ba70 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:35.313751 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357fa0f0-d10a-4c6b-a649-2df4bf01ba70 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:35.392697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa399bbe-fd04-402f-8b44-822dc3bdef21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:35.393400 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa399bbe-fd04-402f-8b44-822dc3bdef21 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:35.393744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa399bbe-fd04-402f-8b44-822dc3bdef21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:35.394030 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa399bbe-fd04-402f-8b44-822dc3bdef21 admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:35.394735 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa399bbe-fd04-402f-8b44-822dc3bdef21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:35.396708 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa399bbe-fd04-402f-8b44-822dc3bdef21 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:35.533700 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fffd5-22e9-4e92-b800-e0ec85e29c2b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:35.534799 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1236/2477] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:40:35 2022] POST /identity/v3/users => generated 294 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:40:35.546992 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fffd5-22e9-4e92-b800-e0ec85e29c2b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:35.553477 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5241cb8-64ab-449d-8ba8-26da9c3b884f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:35.554214 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5241cb8-64ab-449d-8ba8-26da9c3b884f admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:35.554999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5241cb8-64ab-449d-8ba8-26da9c3b884f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:35.555434 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5241cb8-64ab-449d-8ba8-26da9c3b884f admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:35.556762 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5241cb8-64ab-449d-8ba8-26da9c3b884f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:35.558480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5241cb8-64ab-449d-8ba8-26da9c3b884f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:35.577266 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1242/2478] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:40:35.580126 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa399bbe-fd04-402f-8b44-822dc3bdef21 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:35.595223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa399bbe-fd04-402f-8b44-822dc3bdef21 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:35.602973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2979057-b2be-4930-9e05-c025cecea1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:35.614116 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2979057-b2be-4930-9e05-c025cecea1a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:35.614483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2979057-b2be-4930-9e05-c025cecea1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:35.614834 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2979057-b2be-4930-9e05-c025cecea1a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:35.615573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2979057-b2be-4930-9e05-c025cecea1a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:35.617208 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2979057-b2be-4930-9e05-c025cecea1a2 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:35.630214 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1237/2479] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:35 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:40:35.634422 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5241cb8-64ab-449d-8ba8-26da9c3b884f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:35.647963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5241cb8-64ab-449d-8ba8-26da9c3b884f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:35.655032 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a9752-fb0c-4855-9035-3424bcf5c06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:35.656040 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a9752-fb0c-4855-9035-3424bcf5c06b admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:35.656331 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a9752-fb0c-4855-9035-3424bcf5c06b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:35.660733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704a9752-fb0c-4855-9035-3424bcf5c06b admin admin] PATH_INFO: `/v3/projects/c2c8e92805fa41a4a7262e5806ec2d0e/users/bd0b2d7837574730adda51c4a1c44f85/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:35.674213 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a9752-fb0c-4855-9035-3424bcf5c06b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2c8e92805fa41a4a7262e5806ec2d0e, user_id=bd0b2d7837574730adda51c4a1c44f85, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:35.677285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704a9752-fb0c-4855-9035-3424bcf5c06b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:35.786911 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1243/2480] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:40:35 2022] PUT /identity/v3/projects/c2c8e92805fa41a4a7262e5806ec2d0e/users/bd0b2d7837574730adda51c4a1c44f85/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:35.797610 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f113f6-9c44-4dbc-826e-3420977b81b4 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:35.797973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f113f6-9c44-4dbc-826e-3420977b81b4 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:35.798269 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f113f6-9c44-4dbc-826e-3420977b81b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:35.831074 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-44f113f6-9c44-4dbc-826e-3420977b81b4 None None] MFA Rules not processed for user `bd0b2d7837574730adda51c4a1c44f85`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:35.832430 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-44f113f6-9c44-4dbc-826e-3420977b81b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:35.911549 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1238/2481] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:35.912989 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a9752-fb0c-4855-9035-3424bcf5c06b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:35.992289 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704a9752-fb0c-4855-9035-3424bcf5c06b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:35.997420 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20d4e1-ff1e-4e3e-8a14-1db7cece9e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:35.998107 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20d4e1-ff1e-4e3e-8a14-1db7cece9e8f admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:35.998398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20d4e1-ff1e-4e3e-8a14-1db7cece9e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:36.001691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f20d4e1-ff1e-4e3e-8a14-1db7cece9e8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:36.001691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20d4e1-ff1e-4e3e-8a14-1db7cece9e8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:36.002006 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f20d4e1-ff1e-4e3e-8a14-1db7cece9e8f admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:36.093020 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:36.101012 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1244/2482] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:40:35 2022] POST /identity/v3/projects => generated 387 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:40:36.106030 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f113f6-9c44-4dbc-826e-3420977b81b4 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:36.116960 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f113f6-9c44-4dbc-826e-3420977b81b4 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:36.177316 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daabd2d-e481-4a91-9134-f4fe8777ba6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:36.177993 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daabd2d-e481-4a91-9134-f4fe8777ba6a admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:36.178275 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daabd2d-e481-4a91-9134-f4fe8777ba6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:36.178979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daabd2d-e481-4a91-9134-f4fe8777ba6a admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:36.179309 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daabd2d-e481-4a91-9134-f4fe8777ba6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:36.181235 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daabd2d-e481-4a91-9134-f4fe8777ba6a admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:36.288015 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1239/2483] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:40:36 2022] POST /identity/v3/users => generated 300 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:40:36.291957 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20d4e1-ff1e-4e3e-8a14-1db7cece9e8f admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:36.304575 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f20d4e1-ff1e-4e3e-8a14-1db7cece9e8f admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:36.309311 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a511b9-e762-460d-958d-ab9300b375b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:36.309997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a511b9-e762-460d-958d-ab9300b375b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:36.310287 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a511b9-e762-460d-958d-ab9300b375b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:36.310571 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a511b9-e762-460d-958d-ab9300b375b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:36.311340 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a511b9-e762-460d-958d-ab9300b375b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:36.312976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a511b9-e762-460d-958d-ab9300b375b1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:36.324287 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1245/2484] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:40:36.328564 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daabd2d-e481-4a91-9134-f4fe8777ba6a admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:36.339241 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daabd2d-e481-4a91-9134-f4fe8777ba6a admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:36.343916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068bfb15-bc97-4e26-a023-f5303052f69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:36.344903 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068bfb15-bc97-4e26-a023-f5303052f69f admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:36.348826 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068bfb15-bc97-4e26-a023-f5303052f69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:36.348826 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068bfb15-bc97-4e26-a023-f5303052f69f admin admin] PATH_INFO: `/v3/projects/b5e4403491f3476aa1daa0d4c9e588d7/users/e873c4982d1d413b989e0ad1694f4ec2/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:36.363500 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068bfb15-bc97-4e26-a023-f5303052f69f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5e4403491f3476aa1daa0d4c9e588d7, user_id=e873c4982d1d413b989e0ad1694f4ec2, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:36.366337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068bfb15-bc97-4e26-a023-f5303052f69f admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:36.440211 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1240/2485] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:40:36 2022] PUT /identity/v3/projects/b5e4403491f3476aa1daa0d4c9e588d7/users/e873c4982d1d413b989e0ad1694f4ec2/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:36.444080 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a511b9-e762-460d-958d-ab9300b375b1 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:36.453945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a511b9-e762-460d-958d-ab9300b375b1 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:36.511005 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdd6b51-8616-40ee-94b1-84e5db271784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:36.512134 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd6b51-8616-40ee-94b1-84e5db271784 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:36.512134 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd6b51-8616-40ee-94b1-84e5db271784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:36.512332 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdd6b51-8616-40ee-94b1-84e5db271784 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:36.512974 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdd6b51-8616-40ee-94b1-84e5db271784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:36.514588 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdd6b51-8616-40ee-94b1-84e5db271784 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:36.524868 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1246/2486] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:36 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:40:36.529070 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068bfb15-bc97-4e26-a023-f5303052f69f admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:36.540405 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068bfb15-bc97-4e26-a023-f5303052f69f admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:36.545160 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c96867-4aff-4070-b473-d4297c82b66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:36.546036 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c96867-4aff-4070-b473-d4297c82b66c admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:36.546317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c96867-4aff-4070-b473-d4297c82b66c admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:36.546593 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c96867-4aff-4070-b473-d4297c82b66c admin admin] PATH_INFO: `/v3/projects/b5e4403491f3476aa1daa0d4c9e588d7/users/e873c4982d1d413b989e0ad1694f4ec2/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:36.548820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c96867-4aff-4070-b473-d4297c82b66c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5e4403491f3476aa1daa0d4c9e588d7, user_id=e873c4982d1d413b989e0ad1694f4ec2, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:36.551473 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c96867-4aff-4070-b473-d4297c82b66c admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:36.561139 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:36.564192 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1241/2487] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:40:36 2022] PUT /identity/v3/projects/b5e4403491f3476aa1daa0d4c9e588d7/users/e873c4982d1d413b989e0ad1694f4ec2/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:40:36.575356 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204f9a0-d23d-4b1f-8adf-d539dd9e6c30 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:36.575829 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204f9a0-d23d-4b1f-8adf-d539dd9e6c30 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:36.575923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7204f9a0-d23d-4b1f-8adf-d539dd9e6c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:36.602316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7204f9a0-d23d-4b1f-8adf-d539dd9e6c30 None None] MFA Rules not processed for user `e873c4982d1d413b989e0ad1694f4ec2`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:36.603570 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7204f9a0-d23d-4b1f-8adf-d539dd9e6c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:36.679392 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1247/2488] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:36.691471 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c96867-4aff-4070-b473-d4297c82b66c admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:36.694147 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-b3c96867-4aff-4070-b473-d4297c82b66c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:36.763179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c96867-4aff-4070-b473-d4297c82b66c admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:36.767551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eacfb32-82b6-4a8a-95bb-3a71e05d0eb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:36.768276 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacfb32-82b6-4a8a-95bb-3a71e05d0eb5 service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:36.768558 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacfb32-82b6-4a8a-95bb-3a71e05d0eb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:36.768857 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacfb32-82b6-4a8a-95bb-3a71e05d0eb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:36.778472 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eacfb32-82b6-4a8a-95bb-3a71e05d0eb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:36.780009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eacfb32-82b6-4a8a-95bb-3a71e05d0eb5 service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:36.791102 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1242/2489] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:40:36.818886 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204f9a0-d23d-4b1f-8adf-d539dd9e6c30 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:36.867327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:36.881344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7204f9a0-d23d-4b1f-8adf-d539dd9e6c30 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:36.886041 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ab5ff-fa0f-463f-948f-03c555185fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:36.887352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ab5ff-fa0f-463f-948f-03c555185fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:36.887352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ab5ff-fa0f-463f-948f-03c555185fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:36.887537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ab5ff-fa0f-463f-948f-03c555185fd0 admin admin] PATH_INFO: `/v3/users/bd0b2d7837574730adda51c4a1c44f85` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:36.888764 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ab5ff-fa0f-463f-948f-03c555185fd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd0b2d7837574730adda51c4a1c44f85)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:36.890719 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ab5ff-fa0f-463f-948f-03c555185fd0 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:37.077015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f04ab5ff-fa0f-463f-948f-03c555185fd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd0b2d7837574730adda51c4a1c44f85'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:37.077682 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f04ab5ff-fa0f-463f-948f-03c555185fd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd0b2d7837574730adda51c4a1c44f85'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:37.085204 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f04ab5ff-fa0f-463f-948f-03c555185fd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd0b2d7837574730adda51c4a1c44f85'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:37.085854 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f04ab5ff-fa0f-463f-948f-03c555185fd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd0b2d7837574730adda51c4a1c44f85'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:37.193668 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f04ab5ff-fa0f-463f-948f-03c555185fd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd0b2d7837574730adda51c4a1c44f85'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:37.212908 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1248/2490] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:40:36 2022] DELETE /identity/v3/users/bd0b2d7837574730adda51c4a1c44f85 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:37.217523 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053d480-194e-43ab-b28d-00d4c1aa27df None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:37.218022 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053d480-194e-43ab-b28d-00d4c1aa27df None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:37.218154 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053d480-194e-43ab-b28d-00d4c1aa27df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:37.236307 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-1053d480-194e-43ab-b28d-00d4c1aa27df None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:37.237503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-1053d480-194e-43ab-b28d-00d4c1aa27df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:37.295495 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1243/2491] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:37.311259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ab5ff-fa0f-463f-948f-03c555185fd0 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:37.312821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f04ab5ff-fa0f-463f-948f-03c555185fd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:37.371014 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ab5ff-fa0f-463f-948f-03c555185fd0 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:37.374720 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911a9a7-774b-429c-8617-338d56adc1fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:37.375406 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911a9a7-774b-429c-8617-338d56adc1fe service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:37.375692 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911a9a7-774b-429c-8617-338d56adc1fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:37.376518 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911a9a7-774b-429c-8617-338d56adc1fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:37.385786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911a9a7-774b-429c-8617-338d56adc1fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:37.387219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911a9a7-774b-429c-8617-338d56adc1fe service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:37.397715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1249/2492] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:40:37.887540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053d480-194e-43ab-b28d-00d4c1aa27df None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:37.889178 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-1053d480-194e-43ab-b28d-00d4c1aa27df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:37.902260 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053d480-194e-43ab-b28d-00d4c1aa27df None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:37.908754 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ccba1-b499-4b31-81b2-3c581fe02e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:37.909748 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ccba1-b499-4b31-81b2-3c581fe02e77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:37.910136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ccba1-b499-4b31-81b2-3c581fe02e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:37.910850 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ccba1-b499-4b31-81b2-3c581fe02e77 admin admin] PATH_INFO: `/v3/projects/c2c8e92805fa41a4a7262e5806ec2d0e` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:37.912228 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ccba1-b499-4b31-81b2-3c581fe02e77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2c8e92805fa41a4a7262e5806ec2d0e)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:37.925293 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ccba1-b499-4b31-81b2-3c581fe02e77 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:38.031778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:38.036830 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-359ccba1-b499-4b31-81b2-3c581fe02e77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2c8e92805fa41a4a7262e5806ec2d0e', 'request_id': 'req-359ccba1-b499-4b31-81b2-3c581fe02e77'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:38.043057 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-359ccba1-b499-4b31-81b2-3c581fe02e77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2c8e92805fa41a4a7262e5806ec2d0e', 'request_id': 'req-359ccba1-b499-4b31-81b2-3c581fe02e77'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:38.049322 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-359ccba1-b499-4b31-81b2-3c581fe02e77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2c8e92805fa41a4a7262e5806ec2d0e', 'request_id': 'req-359ccba1-b499-4b31-81b2-3c581fe02e77'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:38.101944 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-359ccba1-b499-4b31-81b2-3c581fe02e77 admin admin] The token cache is being invalidate because project c2c8e92805fa41a4a7262e5806ec2d0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:40:38.102230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-359ccba1-b499-4b31-81b2-3c581fe02e77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:38.102859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-359ccba1-b499-4b31-81b2-3c581fe02e77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:38.104464 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1244/2493] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:40:37 2022] DELETE /identity/v3/projects/c2c8e92805fa41a4a7262e5806ec2d0e => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:38.108463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911a9a7-774b-429c-8617-338d56adc1fe service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:38.118800 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911a9a7-774b-429c-8617-338d56adc1fe service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:38.174632 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16008800-06ed-485e-94bc-ef9c7a66ff28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:38.175547 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16008800-06ed-485e-94bc-ef9c7a66ff28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:38.175836 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16008800-06ed-485e-94bc-ef9c7a66ff28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:38.176115 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16008800-06ed-485e-94bc-ef9c7a66ff28 admin admin] PATH_INFO: `/v3/users/e873c4982d1d413b989e0ad1694f4ec2` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:38.177318 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16008800-06ed-485e-94bc-ef9c7a66ff28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e873c4982d1d413b989e0ad1694f4ec2)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:38.179284 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16008800-06ed-485e-94bc-ef9c7a66ff28 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:38.192006 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:38.373452 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-16008800-06ed-485e-94bc-ef9c7a66ff28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e873c4982d1d413b989e0ad1694f4ec2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:38.374107 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-16008800-06ed-485e-94bc-ef9c7a66ff28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e873c4982d1d413b989e0ad1694f4ec2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:38.381250 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-16008800-06ed-485e-94bc-ef9c7a66ff28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e873c4982d1d413b989e0ad1694f4ec2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:38.381890 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-16008800-06ed-485e-94bc-ef9c7a66ff28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e873c4982d1d413b989e0ad1694f4ec2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:38.439566 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-16008800-06ed-485e-94bc-ef9c7a66ff28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e873c4982d1d413b989e0ad1694f4ec2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:38.457553 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1250/2494] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:40:38 2022] DELETE /identity/v3/users/e873c4982d1d413b989e0ad1694f4ec2 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:38.873957 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ccba1-b499-4b31-81b2-3c581fe02e77 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:38.875495 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-359ccba1-b499-4b31-81b2-3c581fe02e77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:38.890584 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ccba1-b499-4b31-81b2-3c581fe02e77 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:38.949688 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7415a04-93d3-42cd-9432-f7cfd3f804b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:38.950562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7415a04-93d3-42cd-9432-f7cfd3f804b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:38.950882 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7415a04-93d3-42cd-9432-f7cfd3f804b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:38.951161 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7415a04-93d3-42cd-9432-f7cfd3f804b4 admin admin] PATH_INFO: `/v3/projects/b5e4403491f3476aa1daa0d4c9e588d7` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:38.952386 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7415a04-93d3-42cd-9432-f7cfd3f804b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5e4403491f3476aa1daa0d4c9e588d7)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:38.955173 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7415a04-93d3-42cd-9432-f7cfd3f804b4 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:39.162169 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-f7415a04-93d3-42cd-9432-f7cfd3f804b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5e4403491f3476aa1daa0d4c9e588d7', 'request_id': 'req-f7415a04-93d3-42cd-9432-f7cfd3f804b4'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:39.168591 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-f7415a04-93d3-42cd-9432-f7cfd3f804b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5e4403491f3476aa1daa0d4c9e588d7', 'request_id': 'req-f7415a04-93d3-42cd-9432-f7cfd3f804b4'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:39.175297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-f7415a04-93d3-42cd-9432-f7cfd3f804b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5e4403491f3476aa1daa0d4c9e588d7', 'request_id': 'req-f7415a04-93d3-42cd-9432-f7cfd3f804b4'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:39.476313 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-f7415a04-93d3-42cd-9432-f7cfd3f804b4 admin admin] The token cache is being invalidate because project b5e4403491f3476aa1daa0d4c9e588d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:40:39.476313 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-f7415a04-93d3-42cd-9432-f7cfd3f804b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:39.476313 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-f7415a04-93d3-42cd-9432-f7cfd3f804b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:39.476313 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1245/2495] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:40:38 2022] DELETE /identity/v3/projects/b5e4403491f3476aa1daa0d4c9e588d7 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:39.479393 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d9cf5-8a31-43fe-b0e3-1056ca9d865f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:39.479918 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d9cf5-8a31-43fe-b0e3-1056ca9d865f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:39.480061 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d9cf5-8a31-43fe-b0e3-1056ca9d865f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:39.506520 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-6e6d9cf5-8a31-43fe-b0e3-1056ca9d865f None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:39.507409 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6e6d9cf5-8a31-43fe-b0e3-1056ca9d865f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:39.569947 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1251/2496] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:39.577860 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae5169-32de-42f8-91db-87d0c3ff20b4 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:39.578150 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae5169-32de-42f8-91db-87d0c3ff20b4 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:39.578536 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ae5169-32de-42f8-91db-87d0c3ff20b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:39.597517 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-70ae5169-32de-42f8-91db-87d0c3ff20b4 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:39.598741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-70ae5169-32de-42f8-91db-87d0c3ff20b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:39.606755 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1246/2497] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:40:39.611914 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d9cf5-8a31-43fe-b0e3-1056ca9d865f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:39.622584 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d9cf5-8a31-43fe-b0e3-1056ca9d865f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:39.627568 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49b143-defc-45a7-adc4-0a7926d999bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:39.630347 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49b143-defc-45a7-adc4-0a7926d999bc admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:39.630347 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49b143-defc-45a7-adc4-0a7926d999bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:39.630347 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f49b143-defc-45a7-adc4-0a7926d999bc admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:39.630347 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f49b143-defc-45a7-adc4-0a7926d999bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:40:39.630347 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f49b143-defc-45a7-adc4-0a7926d999bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:39.631614 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f49b143-defc-45a7-adc4-0a7926d999bc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:39.643737 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1252/2498] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:40:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:40:39.647957 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae5169-32de-42f8-91db-87d0c3ff20b4 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:39.657739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ae5169-32de-42f8-91db-87d0c3ff20b4 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:39.662768 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d3c88-90e9-4b3b-8561-d782ce8ea176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:39.663451 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d3c88-90e9-4b3b-8561-d782ce8ea176 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:39.663735 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d3c88-90e9-4b3b-8561-d782ce8ea176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:39.664013 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106d3c88-90e9-4b3b-8561-d782ce8ea176 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:39.664701 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106d3c88-90e9-4b3b-8561-d782ce8ea176 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:39.666612 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106d3c88-90e9-4b3b-8561-d782ce8ea176 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:39.842763 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1247/2499] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:40:39 2022] POST /identity/v3/projects => generated 391 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:40:39.847417 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49b143-defc-45a7-adc4-0a7926d999bc admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:39.857329 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f49b143-defc-45a7-adc4-0a7926d999bc admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:39.913841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93053b69-40bb-45e5-a374-7ec08279dea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:39.914512 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93053b69-40bb-45e5-a374-7ec08279dea8 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:39.914798 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93053b69-40bb-45e5-a374-7ec08279dea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:39.915083 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93053b69-40bb-45e5-a374-7ec08279dea8 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:39.915818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93053b69-40bb-45e5-a374-7ec08279dea8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:39.917769 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93053b69-40bb-45e5-a374-7ec08279dea8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:40.061153 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1253/2500] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:40:39 2022] POST /identity/v3/users => generated 296 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:40:40.062543 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d3c88-90e9-4b3b-8561-d782ce8ea176 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:40.073503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106d3c88-90e9-4b3b-8561-d782ce8ea176 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:40.080523 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19ebeb-a7cd-4c13-a91f-7124be471705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:40.080523 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19ebeb-a7cd-4c13-a91f-7124be471705 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:40.082040 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19ebeb-a7cd-4c13-a91f-7124be471705 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:40.082040 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19ebeb-a7cd-4c13-a91f-7124be471705 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:40.082040 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f19ebeb-a7cd-4c13-a91f-7124be471705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:40.088109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f19ebeb-a7cd-4c13-a91f-7124be471705 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:40.096521 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1248/2501] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:40:40.100469 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93053b69-40bb-45e5-a374-7ec08279dea8 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:40.113194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93053b69-40bb-45e5-a374-7ec08279dea8 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:40.117794 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177b9754-b6df-42c7-aa07-ede5d0659440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:40.118522 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b9754-b6df-42c7-aa07-ede5d0659440 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:40.118814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b9754-b6df-42c7-aa07-ede5d0659440 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:40.119095 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177b9754-b6df-42c7-aa07-ede5d0659440 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:40.120043 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b9754-b6df-42c7-aa07-ede5d0659440 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:40.121670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177b9754-b6df-42c7-aa07-ede5d0659440 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:40.123093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:40.136379 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1254/2502] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:40:40.139128 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19ebeb-a7cd-4c13-a91f-7124be471705 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:40.152446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19ebeb-a7cd-4c13-a91f-7124be471705 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:40.156012 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e65271-0cf5-498e-a1fc-54212333cb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:40.156910 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e65271-0cf5-498e-a1fc-54212333cb26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:40.157782 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e65271-0cf5-498e-a1fc-54212333cb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:40.158155 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e65271-0cf5-498e-a1fc-54212333cb26 admin admin] PATH_INFO: `/v3/projects/7bda82d2353444048d8384eb5d6af611/users/dcf5d3decd994fe3ba39f8e5240ca2d5/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:40.175054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e65271-0cf5-498e-a1fc-54212333cb26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bda82d2353444048d8384eb5d6af611, user_id=dcf5d3decd994fe3ba39f8e5240ca2d5, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:40.177906 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e65271-0cf5-498e-a1fc-54212333cb26 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:40.361130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:40.368137 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1249/2503] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:40:40 2022] PUT /identity/v3/projects/7bda82d2353444048d8384eb5d6af611/users/dcf5d3decd994fe3ba39f8e5240ca2d5/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:40.382360 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca14ccc-174e-4bd4-9586-1ec7b23b7a70 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:40.382360 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca14ccc-174e-4bd4-9586-1ec7b23b7a70 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:40.382360 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca14ccc-174e-4bd4-9586-1ec7b23b7a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:40.410097 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0ca14ccc-174e-4bd4-9586-1ec7b23b7a70 None None] MFA Rules not processed for user `dcf5d3decd994fe3ba39f8e5240ca2d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:40.411727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0ca14ccc-174e-4bd4-9586-1ec7b23b7a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:40.476864 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1255/2504] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:40.481542 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e65271-0cf5-498e-a1fc-54212333cb26 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:40.568872 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e65271-0cf5-498e-a1fc-54212333cb26 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:40.574280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ddaad-1af4-4301-8206-444acd8b107c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:40.575026 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ddaad-1af4-4301-8206-444acd8b107c admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:40.575335 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ddaad-1af4-4301-8206-444acd8b107c admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:40.575714 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470ddaad-1af4-4301-8206-444acd8b107c admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:40.576429 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ddaad-1af4-4301-8206-444acd8b107c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:40.578343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470ddaad-1af4-4301-8206-444acd8b107c admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:40.721338 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1250/2505] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:40:40 2022] POST /identity/v3/projects => generated 393 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:40:40.725766 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca14ccc-174e-4bd4-9586-1ec7b23b7a70 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:40.737187 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca14ccc-174e-4bd4-9586-1ec7b23b7a70 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:40.793675 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650ca5d-5f7f-44c3-9650-44f987c607de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:40.794351 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650ca5d-5f7f-44c3-9650-44f987c607de admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:40.794662 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650ca5d-5f7f-44c3-9650-44f987c607de admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:40.794967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9650ca5d-5f7f-44c3-9650-44f987c607de admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:40.795740 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9650ca5d-5f7f-44c3-9650-44f987c607de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:40.798446 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9650ca5d-5f7f-44c3-9650-44f987c607de admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:40.936675 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1256/2506] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:40:40 2022] POST /identity/v3/users => generated 303 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:40:40.941286 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ddaad-1af4-4301-8206-444acd8b107c admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:40.951444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470ddaad-1af4-4301-8206-444acd8b107c admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:40.956145 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847e689-e2d2-48ea-aa91-e979c290a0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:40.957631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847e689-e2d2-48ea-aa91-e979c290a0c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:40.957631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847e689-e2d2-48ea-aa91-e979c290a0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:40.957631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847e689-e2d2-48ea-aa91-e979c290a0c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:40.958063 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0847e689-e2d2-48ea-aa91-e979c290a0c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:40.959706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0847e689-e2d2-48ea-aa91-e979c290a0c2 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:40.970170 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1251/2507] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:40:40.974312 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650ca5d-5f7f-44c3-9650-44f987c607de admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:40.984567 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9650ca5d-5f7f-44c3-9650-44f987c607de admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:40.990304 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b4872-b6c2-4092-b74a-41092c7fd9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:40.991182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b4872-b6c2-4092-b74a-41092c7fd9f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:40.991497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b4872-b6c2-4092-b74a-41092c7fd9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:40.991770 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2b4872-b6c2-4092-b74a-41092c7fd9f2 admin admin] PATH_INFO: `/v3/projects/f4ca687929cb46b5b3a32c4d5e5ceeef/users/c10d8fffe7b7464aa9238950b681f892/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:41.007820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b4872-b6c2-4092-b74a-41092c7fd9f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4ca687929cb46b5b3a32c4d5e5ceeef, user_id=c10d8fffe7b7464aa9238950b681f892, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:41.010591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2b4872-b6c2-4092-b74a-41092c7fd9f2 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:41.131355 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1257/2508] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:40:40 2022] PUT /identity/v3/projects/f4ca687929cb46b5b3a32c4d5e5ceeef/users/c10d8fffe7b7464aa9238950b681f892/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:41.135430 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847e689-e2d2-48ea-aa91-e979c290a0c2 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:41.145548 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847e689-e2d2-48ea-aa91-e979c290a0c2 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:41.204575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e548b494-8806-41b6-909d-1be1725a131c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:41.204786 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548b494-8806-41b6-909d-1be1725a131c admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:41.205035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548b494-8806-41b6-909d-1be1725a131c admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:41.205311 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e548b494-8806-41b6-909d-1be1725a131c admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:41.206032 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548b494-8806-41b6-909d-1be1725a131c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:41.207778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e548b494-8806-41b6-909d-1be1725a131c admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:41.219324 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1252/2509] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:41 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:40:41.222243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b4872-b6c2-4092-b74a-41092c7fd9f2 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:41.232989 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2b4872-b6c2-4092-b74a-41092c7fd9f2 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:41.238320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28802c0f-c33d-4b88-8987-25aa4b6684c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:41.239240 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28802c0f-c33d-4b88-8987-25aa4b6684c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:41.239572 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28802c0f-c33d-4b88-8987-25aa4b6684c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:41.239852 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28802c0f-c33d-4b88-8987-25aa4b6684c8 admin admin] PATH_INFO: `/v3/projects/f4ca687929cb46b5b3a32c4d5e5ceeef/users/c10d8fffe7b7464aa9238950b681f892/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:41.241822 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28802c0f-c33d-4b88-8987-25aa4b6684c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4ca687929cb46b5b3a32c4d5e5ceeef, user_id=c10d8fffe7b7464aa9238950b681f892, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:41.244537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28802c0f-c33d-4b88-8987-25aa4b6684c8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:41.256148 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1258/2510] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:40:41 2022] PUT /identity/v3/projects/f4ca687929cb46b5b3a32c4d5e5ceeef/users/c10d8fffe7b7464aa9238950b681f892/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:40:41.267596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03284978-68d2-4538-b645-d5e0ff3cfdb3 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:41.268151 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03284978-68d2-4538-b645-d5e0ff3cfdb3 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:41.268438 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03284978-68d2-4538-b645-d5e0ff3cfdb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:41.297321 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-03284978-68d2-4538-b645-d5e0ff3cfdb3 None None] MFA Rules not processed for user `c10d8fffe7b7464aa9238950b681f892`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:41.299140 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-03284978-68d2-4538-b645-d5e0ff3cfdb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:41.374055 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:41.390471 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1253/2511] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:41 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:41.399493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28802c0f-c33d-4b88-8987-25aa4b6684c8 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:41.401226 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-28802c0f-c33d-4b88-8987-25aa4b6684c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:41.453766 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:41.485547 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28802c0f-c33d-4b88-8987-25aa4b6684c8 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:41.485547 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775069a4-acf7-4ca9-9a5b-e05c87ae8f94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:41.485547 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775069a4-acf7-4ca9-9a5b-e05c87ae8f94 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:41.485547 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775069a4-acf7-4ca9-9a5b-e05c87ae8f94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:41.486069 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775069a4-acf7-4ca9-9a5b-e05c87ae8f94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:41.501991 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775069a4-acf7-4ca9-9a5b-e05c87ae8f94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:41.503697 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775069a4-acf7-4ca9-9a5b-e05c87ae8f94 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:41.520175 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1259/2512] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:41 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:40:41.571810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03284978-68d2-4538-b645-d5e0ff3cfdb3 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:41.577383 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-03284978-68d2-4538-b645-d5e0ff3cfdb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:41.590884 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03284978-68d2-4538-b645-d5e0ff3cfdb3 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:41.605450 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cfd8dd-158f-4519-a883-8d4f28089e7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:41.606134 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfd8dd-158f-4519-a883-8d4f28089e7f service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:41.606425 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfd8dd-158f-4519-a883-8d4f28089e7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:41.606729 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cfd8dd-158f-4519-a883-8d4f28089e7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:41.725083 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cfd8dd-158f-4519-a883-8d4f28089e7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:41.734760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775069a4-acf7-4ca9-9a5b-e05c87ae8f94 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:41.739316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-775069a4-acf7-4ca9-9a5b-e05c87ae8f94 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:41.739500 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cfd8dd-158f-4519-a883-8d4f28089e7f service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:41.767715 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1254/2513] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:40:41.844568 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775069a4-acf7-4ca9-9a5b-e05c87ae8f94 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:41.852836 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b58d96-25b0-4e25-a611-f43dbb6503cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:41.853027 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b58d96-25b0-4e25-a611-f43dbb6503cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:41.853476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b58d96-25b0-4e25-a611-f43dbb6503cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:41.853904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b58d96-25b0-4e25-a611-f43dbb6503cc admin admin] PATH_INFO: `/v3/users/fa1c093874a74439ba62aa051946e52a` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:41.856181 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b58d96-25b0-4e25-a611-f43dbb6503cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa1c093874a74439ba62aa051946e52a)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:41.859557 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b58d96-25b0-4e25-a611-f43dbb6503cc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:42.077606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-22b58d96-25b0-4e25-a611-f43dbb6503cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa1c093874a74439ba62aa051946e52a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:42.078303 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-22b58d96-25b0-4e25-a611-f43dbb6503cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa1c093874a74439ba62aa051946e52a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:42.086695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-22b58d96-25b0-4e25-a611-f43dbb6503cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa1c093874a74439ba62aa051946e52a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:42.087386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-22b58d96-25b0-4e25-a611-f43dbb6503cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa1c093874a74439ba62aa051946e52a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:42.143217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-22b58d96-25b0-4e25-a611-f43dbb6503cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa1c093874a74439ba62aa051946e52a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:42.163590 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1260/2514] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:40:41 2022] DELETE /identity/v3/users/fa1c093874a74439ba62aa051946e52a => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:42.168525 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280124b-ec1e-4720-80d6-9c22dc374028 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:42.169052 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280124b-ec1e-4720-80d6-9c22dc374028 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:42.169162 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2280124b-ec1e-4720-80d6-9c22dc374028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:42.187500 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-2280124b-ec1e-4720-80d6-9c22dc374028 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:42.189360 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-2280124b-ec1e-4720-80d6-9c22dc374028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:42.253847 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1255/2515] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:42.263417 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b58d96-25b0-4e25-a611-f43dbb6503cc admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:42.265035 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-22b58d96-25b0-4e25-a611-f43dbb6503cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:42.324510 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b58d96-25b0-4e25-a611-f43dbb6503cc admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:42.328153 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229da213-b495-4911-99ee-9addb09d0dab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:42.328739 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229da213-b495-4911-99ee-9addb09d0dab service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:42.329046 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229da213-b495-4911-99ee-9addb09d0dab service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:42.329311 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229da213-b495-4911-99ee-9addb09d0dab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:42.338410 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229da213-b495-4911-99ee-9addb09d0dab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:42.339978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229da213-b495-4911-99ee-9addb09d0dab service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:42.361164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1261/2516] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:40:42.967176 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280124b-ec1e-4720-80d6-9c22dc374028 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:42.968899 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-2280124b-ec1e-4720-80d6-9c22dc374028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:42.982190 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2280124b-ec1e-4720-80d6-9c22dc374028 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:42.987567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbf142f-254c-4f13-970e-83964ce18538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:42.988530 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf142f-254c-4f13-970e-83964ce18538 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:42.989017 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf142f-254c-4f13-970e-83964ce18538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:42.989463 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbf142f-254c-4f13-970e-83964ce18538 admin admin] PATH_INFO: `/v3/projects/80ba2ca60b7e4578b9334dd6eb013eec` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:42.990688 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf142f-254c-4f13-970e-83964ce18538 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80ba2ca60b7e4578b9334dd6eb013eec)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:42.992721 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fbf142f-254c-4f13-970e-83964ce18538 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:43.141158 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9fbf142f-254c-4f13-970e-83964ce18538 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80ba2ca60b7e4578b9334dd6eb013eec', 'request_id': 'req-9fbf142f-254c-4f13-970e-83964ce18538'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:43.147249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9fbf142f-254c-4f13-970e-83964ce18538 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80ba2ca60b7e4578b9334dd6eb013eec', 'request_id': 'req-9fbf142f-254c-4f13-970e-83964ce18538'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:43.152751 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9fbf142f-254c-4f13-970e-83964ce18538 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80ba2ca60b7e4578b9334dd6eb013eec', 'request_id': 'req-9fbf142f-254c-4f13-970e-83964ce18538'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:43.216077 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9fbf142f-254c-4f13-970e-83964ce18538 admin admin] The token cache is being invalidate because project 80ba2ca60b7e4578b9334dd6eb013eec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:40:43.216694 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9fbf142f-254c-4f13-970e-83964ce18538 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:43.217038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9fbf142f-254c-4f13-970e-83964ce18538 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:43.218591 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1256/2517] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:40:42 2022] DELETE /identity/v3/projects/80ba2ca60b7e4578b9334dd6eb013eec => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:43.223671 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0153c-7a2a-46c7-89c5-307bccef7bfc None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:43.224107 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0153c-7a2a-46c7-89c5-307bccef7bfc None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:43.224240 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af0153c-7a2a-46c7-89c5-307bccef7bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:43.230495 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:43.244065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4af0153c-7a2a-46c7-89c5-307bccef7bfc None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:43.248934 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4af0153c-7a2a-46c7-89c5-307bccef7bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:43.306705 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1262/2518] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:43.314650 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7b094-a6ed-4bef-82ce-f8eea82f86e0 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:43.314939 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7b094-a6ed-4bef-82ce-f8eea82f86e0 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:43.315218 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a7b094-a6ed-4bef-82ce-f8eea82f86e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:43.333144 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-07a7b094-a6ed-4bef-82ce-f8eea82f86e0 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:43.334341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-07a7b094-a6ed-4bef-82ce-f8eea82f86e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:43.342472 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1257/2519] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:43.347201 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0153c-7a2a-46c7-89c5-307bccef7bfc None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:43.358266 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af0153c-7a2a-46c7-89c5-307bccef7bfc None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:43.364010 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad2bc8-8d5d-4a12-b3ea-fb756bc793e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:43.364727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad2bc8-8d5d-4a12-b3ea-fb756bc793e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:43.365010 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad2bc8-8d5d-4a12-b3ea-fb756bc793e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:43.365285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ad2bc8-8d5d-4a12-b3ea-fb756bc793e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:43.366037 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad2bc8-8d5d-4a12-b3ea-fb756bc793e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:40:43.366342 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad2bc8-8d5d-4a12-b3ea-fb756bc793e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:43.367964 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ad2bc8-8d5d-4a12-b3ea-fb756bc793e0 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:43.380286 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1263/2520] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:40:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:40:43.383948 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7b094-a6ed-4bef-82ce-f8eea82f86e0 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:43.393272 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a7b094-a6ed-4bef-82ce-f8eea82f86e0 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:43.397904 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34df7ef-47fd-4699-a5cc-2496122151e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:43.398577 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34df7ef-47fd-4699-a5cc-2496122151e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:43.398860 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34df7ef-47fd-4699-a5cc-2496122151e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:43.399133 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34df7ef-47fd-4699-a5cc-2496122151e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:43.399874 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34df7ef-47fd-4699-a5cc-2496122151e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:43.401758 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34df7ef-47fd-4699-a5cc-2496122151e7 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:43.405901 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:43.533354 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1258/2521] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:40:43 2022] POST /identity/v3/projects => generated 365 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:40:43.538679 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad2bc8-8d5d-4a12-b3ea-fb756bc793e0 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:43.553416 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ad2bc8-8d5d-4a12-b3ea-fb756bc793e0 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:43.609679 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2be454-0547-42a8-adb9-35bc3ea45215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:43.610959 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2be454-0547-42a8-adb9-35bc3ea45215 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:43.611220 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2be454-0547-42a8-adb9-35bc3ea45215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:43.611523 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2be454-0547-42a8-adb9-35bc3ea45215 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:43.612218 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2be454-0547-42a8-adb9-35bc3ea45215 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:43.614227 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2be454-0547-42a8-adb9-35bc3ea45215 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:43.699775 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1264/2522] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:40:43 2022] POST /identity/v3/users => generated 283 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:40:43.702176 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34df7ef-47fd-4699-a5cc-2496122151e7 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:43.713393 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34df7ef-47fd-4699-a5cc-2496122151e7 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:43.718018 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cddc28-5435-4965-a196-d7c132175dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:43.718729 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cddc28-5435-4965-a196-d7c132175dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:43.719037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cddc28-5435-4965-a196-d7c132175dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:43.719343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cddc28-5435-4965-a196-d7c132175dd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:43.720739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cddc28-5435-4965-a196-d7c132175dd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:43.722321 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cddc28-5435-4965-a196-d7c132175dd3 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:43.733559 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1259/2523] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:40:43.737257 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2be454-0547-42a8-adb9-35bc3ea45215 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:43.748147 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2be454-0547-42a8-adb9-35bc3ea45215 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:43.752895 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dd3495-7190-4eba-9832-bffdcceb9895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:43.753671 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd3495-7190-4eba-9832-bffdcceb9895 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:43.753962 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd3495-7190-4eba-9832-bffdcceb9895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:43.754239 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dd3495-7190-4eba-9832-bffdcceb9895 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:43.754974 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd3495-7190-4eba-9832-bffdcceb9895 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:43.756623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dd3495-7190-4eba-9832-bffdcceb9895 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:43.767166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1265/2524] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:40:43.772914 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cddc28-5435-4965-a196-d7c132175dd3 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:43.791944 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cddc28-5435-4965-a196-d7c132175dd3 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:43.796698 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6fc94a-a508-448e-8808-616673af0891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:43.797631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6fc94a-a508-448e-8808-616673af0891 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:43.797902 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6fc94a-a508-448e-8808-616673af0891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:43.798184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6fc94a-a508-448e-8808-616673af0891 admin admin] PATH_INFO: `/v3/projects/f4c9c588fdb54c10be72ad24b8465995/users/b9bff5929e654b41aff1d27a99981840/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:43.813031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6fc94a-a508-448e-8808-616673af0891 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4c9c588fdb54c10be72ad24b8465995, user_id=b9bff5929e654b41aff1d27a99981840, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:43.815879 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6fc94a-a508-448e-8808-616673af0891 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:43.898397 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1260/2525] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:40:43 2022] PUT /identity/v3/projects/f4c9c588fdb54c10be72ad24b8465995/users/b9bff5929e654b41aff1d27a99981840/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:43.910248 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf63e1-c067-482f-b06b-b547d7aa0b3a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:43.910540 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf63e1-c067-482f-b06b-b547d7aa0b3a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:43.911085 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf63e1-c067-482f-b06b-b547d7aa0b3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:43.937973 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e5bf63e1-c067-482f-b06b-b547d7aa0b3a None None] MFA Rules not processed for user `b9bff5929e654b41aff1d27a99981840`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:43.939319 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e5bf63e1-c067-482f-b06b-b547d7aa0b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:44.004636 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:44.010136 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1266/2526] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:43 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:44.024272 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6fc94a-a508-448e-8808-616673af0891 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:44.033161 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-6d6fc94a-a508-448e-8808-616673af0891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:44.091637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6fc94a-a508-448e-8808-616673af0891 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:44.096368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8f16a-09cc-48a1-a1e2-d48e395902c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:44.097100 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8f16a-09cc-48a1-a1e2-d48e395902c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:44.097378 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8f16a-09cc-48a1-a1e2-d48e395902c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:44.097653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8f16a-09cc-48a1-a1e2-d48e395902c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:44.114173 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe8f16a-09cc-48a1-a1e2-d48e395902c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:44.119046 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe8f16a-09cc-48a1-a1e2-d48e395902c4 service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:44.125085 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1261/2527] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:44 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:40:46.681184 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf63e1-c067-482f-b06b-b547d7aa0b3a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:46.682674 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e5bf63e1-c067-482f-b06b-b547d7aa0b3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:46.747832 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf63e1-c067-482f-b06b-b547d7aa0b3a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:46.753022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ed9e9-2267-46c4-9fb4-1b2e49b9fb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:46.753894 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ed9e9-2267-46c4-9fb4-1b2e49b9fb25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:46.754174 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ed9e9-2267-46c4-9fb4-1b2e49b9fb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:46.754445 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74ed9e9-2267-46c4-9fb4-1b2e49b9fb25 admin admin] PATH_INFO: `/v3/users/dcf5d3decd994fe3ba39f8e5240ca2d5` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:46.755716 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ed9e9-2267-46c4-9fb4-1b2e49b9fb25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcf5d3decd994fe3ba39f8e5240ca2d5)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:46.757755 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74ed9e9-2267-46c4-9fb4-1b2e49b9fb25 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:47.022653 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f74ed9e9-2267-46c4-9fb4-1b2e49b9fb25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcf5d3decd994fe3ba39f8e5240ca2d5'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:47.023253 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f74ed9e9-2267-46c4-9fb4-1b2e49b9fb25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcf5d3decd994fe3ba39f8e5240ca2d5'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:47.030172 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f74ed9e9-2267-46c4-9fb4-1b2e49b9fb25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcf5d3decd994fe3ba39f8e5240ca2d5'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:47.030863 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f74ed9e9-2267-46c4-9fb4-1b2e49b9fb25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcf5d3decd994fe3ba39f8e5240ca2d5'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:47.134690 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f74ed9e9-2267-46c4-9fb4-1b2e49b9fb25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcf5d3decd994fe3ba39f8e5240ca2d5'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:47.156788 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1267/2528] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:40:46 2022] DELETE /identity/v3/users/dcf5d3decd994fe3ba39f8e5240ca2d5 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:47.162449 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f150709-f490-4b5d-8459-411e78c68cb1 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:47.162962 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f150709-f490-4b5d-8459-411e78c68cb1 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:47.163086 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f150709-f490-4b5d-8459-411e78c68cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:47.181778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-7f150709-f490-4b5d-8459-411e78c68cb1 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:47.183022 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-7f150709-f490-4b5d-8459-411e78c68cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:47.242531 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1262/2529] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:47.252278 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ed9e9-2267-46c4-9fb4-1b2e49b9fb25 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:47.253729 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f74ed9e9-2267-46c4-9fb4-1b2e49b9fb25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:47.310600 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74ed9e9-2267-46c4-9fb4-1b2e49b9fb25 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:47.314071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69671df-b321-47a6-9355-ec72ee278747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:47.314712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69671df-b321-47a6-9355-ec72ee278747 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:47.314994 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69671df-b321-47a6-9355-ec72ee278747 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:47.315285 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69671df-b321-47a6-9355-ec72ee278747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:47.324186 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69671df-b321-47a6-9355-ec72ee278747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:47.325609 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69671df-b321-47a6-9355-ec72ee278747 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:47.335053 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1268/2530] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:40:47.801770 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f150709-f490-4b5d-8459-411e78c68cb1 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:47.804604 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-7f150709-f490-4b5d-8459-411e78c68cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:47.822847 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f150709-f490-4b5d-8459-411e78c68cb1 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:47.829625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f79f16-4f0d-4c0b-9003-0ad06640f9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:47.830686 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f79f16-4f0d-4c0b-9003-0ad06640f9dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:47.830969 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f79f16-4f0d-4c0b-9003-0ad06640f9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:47.831382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f79f16-4f0d-4c0b-9003-0ad06640f9dd admin admin] PATH_INFO: `/v3/projects/7bda82d2353444048d8384eb5d6af611` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:47.833038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f79f16-4f0d-4c0b-9003-0ad06640f9dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bda82d2353444048d8384eb5d6af611)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:47.836575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f79f16-4f0d-4c0b-9003-0ad06640f9dd admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:47.935069 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d4f79f16-4f0d-4c0b-9003-0ad06640f9dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bda82d2353444048d8384eb5d6af611', 'request_id': 'req-d4f79f16-4f0d-4c0b-9003-0ad06640f9dd'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:47.936235 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:47.942983 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d4f79f16-4f0d-4c0b-9003-0ad06640f9dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bda82d2353444048d8384eb5d6af611', 'request_id': 'req-d4f79f16-4f0d-4c0b-9003-0ad06640f9dd'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:47.948581 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d4f79f16-4f0d-4c0b-9003-0ad06640f9dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bda82d2353444048d8384eb5d6af611', 'request_id': 'req-d4f79f16-4f0d-4c0b-9003-0ad06640f9dd'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:48.115162 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d4f79f16-4f0d-4c0b-9003-0ad06640f9dd admin admin] The token cache is being invalidate because project 7bda82d2353444048d8384eb5d6af611 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:40:48.115518 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d4f79f16-4f0d-4c0b-9003-0ad06640f9dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:48.116142 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d4f79f16-4f0d-4c0b-9003-0ad06640f9dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:48.117729 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1263/2531] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:40:47 2022] DELETE /identity/v3/projects/7bda82d2353444048d8384eb5d6af611 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:48.121962 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69671df-b321-47a6-9355-ec72ee278747 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:48.133497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69671df-b321-47a6-9355-ec72ee278747 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:48.150712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:48.191784 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5131a4-b0a8-4887-a5fd-61f2e190794b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:48.192648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5131a4-b0a8-4887-a5fd-61f2e190794b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:48.192924 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5131a4-b0a8-4887-a5fd-61f2e190794b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:48.193195 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5131a4-b0a8-4887-a5fd-61f2e190794b admin admin] PATH_INFO: `/v3/users/c10d8fffe7b7464aa9238950b681f892` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:48.194369 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5131a4-b0a8-4887-a5fd-61f2e190794b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c10d8fffe7b7464aa9238950b681f892)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:48.196372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5131a4-b0a8-4887-a5fd-61f2e190794b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:48.389010 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2f5131a4-b0a8-4887-a5fd-61f2e190794b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c10d8fffe7b7464aa9238950b681f892'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:48.389667 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2f5131a4-b0a8-4887-a5fd-61f2e190794b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c10d8fffe7b7464aa9238950b681f892'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:48.397082 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2f5131a4-b0a8-4887-a5fd-61f2e190794b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c10d8fffe7b7464aa9238950b681f892'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:48.397793 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2f5131a4-b0a8-4887-a5fd-61f2e190794b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c10d8fffe7b7464aa9238950b681f892'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:48.468217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2f5131a4-b0a8-4887-a5fd-61f2e190794b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c10d8fffe7b7464aa9238950b681f892'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:48.487293 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1269/2532] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:40:48 2022] DELETE /identity/v3/users/c10d8fffe7b7464aa9238950b681f892 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:48.955890 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f79f16-4f0d-4c0b-9003-0ad06640f9dd admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:48.957622 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d4f79f16-4f0d-4c0b-9003-0ad06640f9dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:48.971576 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f79f16-4f0d-4c0b-9003-0ad06640f9dd admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:49.029055 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04649c7f-72a2-4106-821f-a2ce5b70bd3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:49.029983 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04649c7f-72a2-4106-821f-a2ce5b70bd3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:49.030203 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04649c7f-72a2-4106-821f-a2ce5b70bd3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:49.030481 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04649c7f-72a2-4106-821f-a2ce5b70bd3f admin admin] PATH_INFO: `/v3/projects/f4ca687929cb46b5b3a32c4d5e5ceeef` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:49.031694 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04649c7f-72a2-4106-821f-a2ce5b70bd3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4ca687929cb46b5b3a32c4d5e5ceeef)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:49.034234 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04649c7f-72a2-4106-821f-a2ce5b70bd3f admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:49.300703 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-04649c7f-72a2-4106-821f-a2ce5b70bd3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4ca687929cb46b5b3a32c4d5e5ceeef', 'request_id': 'req-04649c7f-72a2-4106-821f-a2ce5b70bd3f'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:49.307205 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-04649c7f-72a2-4106-821f-a2ce5b70bd3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4ca687929cb46b5b3a32c4d5e5ceeef', 'request_id': 'req-04649c7f-72a2-4106-821f-a2ce5b70bd3f'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:49.313009 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-04649c7f-72a2-4106-821f-a2ce5b70bd3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4ca687929cb46b5b3a32c4d5e5ceeef', 'request_id': 'req-04649c7f-72a2-4106-821f-a2ce5b70bd3f'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:49.390741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-04649c7f-72a2-4106-821f-a2ce5b70bd3f admin admin] The token cache is being invalidate because project f4ca687929cb46b5b3a32c4d5e5ceeef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:40:49.392060 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-04649c7f-72a2-4106-821f-a2ce5b70bd3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:49.392266 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-04649c7f-72a2-4106-821f-a2ce5b70bd3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:49.394057 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1264/2533] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:40:48 2022] DELETE /identity/v3/projects/f4ca687929cb46b5b3a32c4d5e5ceeef => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:49.401168 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4b20e-18bf-4371-880d-bbec78fdd8a4 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:49.401168 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4b20e-18bf-4371-880d-bbec78fdd8a4 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:49.401168 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced4b20e-18bf-4371-880d-bbec78fdd8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:49.418376 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ced4b20e-18bf-4371-880d-bbec78fdd8a4 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:49.419651 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ced4b20e-18bf-4371-880d-bbec78fdd8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:49.482647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1270/2534] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:49.490740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9395bf-6964-4919-90ae-55a973ddaafd None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:49.491526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9395bf-6964-4919-90ae-55a973ddaafd None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:49.492407 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9395bf-6964-4919-90ae-55a973ddaafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:49.517166 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-6f9395bf-6964-4919-90ae-55a973ddaafd None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:49.518873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-6f9395bf-6964-4919-90ae-55a973ddaafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:49.529225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1265/2535] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:49.531895 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4b20e-18bf-4371-880d-bbec78fdd8a4 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:49.542007 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced4b20e-18bf-4371-880d-bbec78fdd8a4 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:49.546533 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c179c8-f638-43b5-8204-5e8afbc88420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:49.547196 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c179c8-f638-43b5-8204-5e8afbc88420 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:49.548012 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c179c8-f638-43b5-8204-5e8afbc88420 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:49.548503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c179c8-f638-43b5-8204-5e8afbc88420 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:49.549358 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c179c8-f638-43b5-8204-5e8afbc88420 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:40:49.549673 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c179c8-f638-43b5-8204-5e8afbc88420 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:49.551310 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c179c8-f638-43b5-8204-5e8afbc88420 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:49.564681 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1271/2536] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:40:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:40:49.569184 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9395bf-6964-4919-90ae-55a973ddaafd None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:49.579382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9395bf-6964-4919-90ae-55a973ddaafd None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:49.584398 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92f57e-29b3-49da-b3e8-fe89497ee069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:49.585066 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92f57e-29b3-49da-b3e8-fe89497ee069 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:49.585355 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92f57e-29b3-49da-b3e8-fe89497ee069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:49.585639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee92f57e-29b3-49da-b3e8-fe89497ee069 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:49.586349 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee92f57e-29b3-49da-b3e8-fe89497ee069 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:49.588325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee92f57e-29b3-49da-b3e8-fe89497ee069 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:49.696183 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1266/2537] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:40:49 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:40:49.697726 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c179c8-f638-43b5-8204-5e8afbc88420 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:49.708173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c179c8-f638-43b5-8204-5e8afbc88420 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:49.735546 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:49.765528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d082b-17b8-4dca-bd87-a609bb2eac60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:49.766233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d082b-17b8-4dca-bd87-a609bb2eac60 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:49.766523 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d082b-17b8-4dca-bd87-a609bb2eac60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:49.766806 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650d082b-17b8-4dca-bd87-a609bb2eac60 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:49.767543 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650d082b-17b8-4dca-bd87-a609bb2eac60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:49.769493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650d082b-17b8-4dca-bd87-a609bb2eac60 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:49.864634 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1272/2538] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:40:49 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:40:49.869255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92f57e-29b3-49da-b3e8-fe89497ee069 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:49.888099 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee92f57e-29b3-49da-b3e8-fe89497ee069 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:49.895842 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0145b859-6ac1-4f46-97d8-8953efc25cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:49.899249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145b859-6ac1-4f46-97d8-8953efc25cbc admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:49.899249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145b859-6ac1-4f46-97d8-8953efc25cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:49.899249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145b859-6ac1-4f46-97d8-8953efc25cbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:49.899249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0145b859-6ac1-4f46-97d8-8953efc25cbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:49.905934 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0145b859-6ac1-4f46-97d8-8953efc25cbc admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:49.916707 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1267/2539] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:40:49.920777 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d082b-17b8-4dca-bd87-a609bb2eac60 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:49.932983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650d082b-17b8-4dca-bd87-a609bb2eac60 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:49.947623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b985e980-205a-4d68-9a96-13437e0c27b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:49.947623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985e980-205a-4d68-9a96-13437e0c27b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:49.947623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985e980-205a-4d68-9a96-13437e0c27b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:49.947623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985e980-205a-4d68-9a96-13437e0c27b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:49.947983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b985e980-205a-4d68-9a96-13437e0c27b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:49.949473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b985e980-205a-4d68-9a96-13437e0c27b2 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:49.960804 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1273/2540] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:49 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:40:49.964975 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0145b859-6ac1-4f46-97d8-8953efc25cbc admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:49.977144 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0145b859-6ac1-4f46-97d8-8953efc25cbc admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:49.982407 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d25cb-a324-472d-836c-e45ce38107c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:49.983321 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d25cb-a324-472d-836c-e45ce38107c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:49.983605 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d25cb-a324-472d-836c-e45ce38107c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:49.983879 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1d25cb-a324-472d-836c-e45ce38107c3 admin admin] PATH_INFO: `/v3/projects/ad9a9c86adf24a1ab226f770fe6907d9/users/f32f66429de34d348bd44aa123230f64/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:50.003637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d25cb-a324-472d-836c-e45ce38107c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad9a9c86adf24a1ab226f770fe6907d9, user_id=f32f66429de34d348bd44aa123230f64, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:50.006783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1d25cb-a324-472d-836c-e45ce38107c3 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:50.073461 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1268/2541] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:40:49 2022] PUT /identity/v3/projects/ad9a9c86adf24a1ab226f770fe6907d9/users/f32f66429de34d348bd44aa123230f64/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:50.085422 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede02315-42a1-40ae-a632-c08ab8e41c52 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:50.085737 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede02315-42a1-40ae-a632-c08ab8e41c52 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:50.085987 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede02315-42a1-40ae-a632-c08ab8e41c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:50.115971 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-ede02315-42a1-40ae-a632-c08ab8e41c52 None None] MFA Rules not processed for user `f32f66429de34d348bd44aa123230f64`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:50.117181 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ede02315-42a1-40ae-a632-c08ab8e41c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:50.186272 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1274/2542] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:50.191518 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d25cb-a324-472d-836c-e45ce38107c3 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:50.197612 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:50.255851 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1d25cb-a324-472d-836c-e45ce38107c3 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:50.261057 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba661d-2b20-4de1-8c0b-bc38ae6d7a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:50.261715 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba661d-2b20-4de1-8c0b-bc38ae6d7a69 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:50.261997 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba661d-2b20-4de1-8c0b-bc38ae6d7a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:50.262269 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba661d-2b20-4de1-8c0b-bc38ae6d7a69 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:50.262951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba661d-2b20-4de1-8c0b-bc38ae6d7a69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:50.264938 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba661d-2b20-4de1-8c0b-bc38ae6d7a69 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:50.350247 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1269/2543] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:40:50 2022] POST /identity/v3/projects => generated 387 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:40:50.354288 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede02315-42a1-40ae-a632-c08ab8e41c52 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:50.365862 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede02315-42a1-40ae-a632-c08ab8e41c52 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:50.422881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9f4fb-8269-491d-adbb-d8bac17c3fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:50.423599 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9f4fb-8269-491d-adbb-d8bac17c3fd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:50.423888 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9f4fb-8269-491d-adbb-d8bac17c3fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:50.424170 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc9f4fb-8269-491d-adbb-d8bac17c3fd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:50.424881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9f4fb-8269-491d-adbb-d8bac17c3fd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:50.426841 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc9f4fb-8269-491d-adbb-d8bac17c3fd0 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:50.546601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1275/2544] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:40:50 2022] POST /identity/v3/users => generated 300 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:40:50.550293 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba661d-2b20-4de1-8c0b-bc38ae6d7a69 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:50.564037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba661d-2b20-4de1-8c0b-bc38ae6d7a69 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:50.565815 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953746a7-f1b0-43da-bee8-7f95e9e8e5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:50.566609 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953746a7-f1b0-43da-bee8-7f95e9e8e5a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:50.566898 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953746a7-f1b0-43da-bee8-7f95e9e8e5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:50.567178 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953746a7-f1b0-43da-bee8-7f95e9e8e5a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:50.567935 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953746a7-f1b0-43da-bee8-7f95e9e8e5a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:50.569524 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953746a7-f1b0-43da-bee8-7f95e9e8e5a1 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:50.580854 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1270/2545] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:40:50.591991 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9f4fb-8269-491d-adbb-d8bac17c3fd0 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:50.602538 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc9f4fb-8269-491d-adbb-d8bac17c3fd0 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:50.607162 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a7b65b-a3bd-48a2-8707-ca874b1b2205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:50.608074 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7b65b-a3bd-48a2-8707-ca874b1b2205 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:50.608364 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7b65b-a3bd-48a2-8707-ca874b1b2205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:50.608644 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7b65b-a3bd-48a2-8707-ca874b1b2205 admin admin] PATH_INFO: `/v3/projects/1d845010a9fc4ce99410ca30b2436b83/users/785a2703cd6c4ba3a64393098bd99538/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:50.623065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a7b65b-a3bd-48a2-8707-ca874b1b2205 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d845010a9fc4ce99410ca30b2436b83, user_id=785a2703cd6c4ba3a64393098bd99538, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:50.625849 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a7b65b-a3bd-48a2-8707-ca874b1b2205 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:50.732241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:50.737206 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1276/2546] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:40:50 2022] PUT /identity/v3/projects/1d845010a9fc4ce99410ca30b2436b83/users/785a2703cd6c4ba3a64393098bd99538/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:50.741279 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953746a7-f1b0-43da-bee8-7f95e9e8e5a1 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:50.751531 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953746a7-f1b0-43da-bee8-7f95e9e8e5a1 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:50.807609 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9136c7a1-d244-48af-8fab-f5b6d770f42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:50.808203 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136c7a1-d244-48af-8fab-f5b6d770f42f admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:50.808488 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136c7a1-d244-48af-8fab-f5b6d770f42f admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:50.808763 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9136c7a1-d244-48af-8fab-f5b6d770f42f admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:50.809530 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9136c7a1-d244-48af-8fab-f5b6d770f42f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:50.811140 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9136c7a1-d244-48af-8fab-f5b6d770f42f admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:50.821596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1271/2547] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:50 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:40:50.825797 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a7b65b-a3bd-48a2-8707-ca874b1b2205 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:50.837111 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a7b65b-a3bd-48a2-8707-ca874b1b2205 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:50.841681 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e455c-024a-48e9-8af7-93d5dbf9e591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:50.842587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e455c-024a-48e9-8af7-93d5dbf9e591 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:50.842868 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e455c-024a-48e9-8af7-93d5dbf9e591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:50.843144 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e455c-024a-48e9-8af7-93d5dbf9e591 admin admin] PATH_INFO: `/v3/projects/1d845010a9fc4ce99410ca30b2436b83/users/785a2703cd6c4ba3a64393098bd99538/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:50.845526 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6e455c-024a-48e9-8af7-93d5dbf9e591 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d845010a9fc4ce99410ca30b2436b83, user_id=785a2703cd6c4ba3a64393098bd99538, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:50.848233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6e455c-024a-48e9-8af7-93d5dbf9e591 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:50.858514 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1277/2548] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:40:50 2022] PUT /identity/v3/projects/1d845010a9fc4ce99410ca30b2436b83/users/785a2703cd6c4ba3a64393098bd99538/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:50.874293 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348b4c3-91a5-4979-8340-552f2ef0154d None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:50.874589 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348b4c3-91a5-4979-8340-552f2ef0154d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:50.874875 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348b4c3-91a5-4979-8340-552f2ef0154d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:50.902357 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-5348b4c3-91a5-4979-8340-552f2ef0154d None None] MFA Rules not processed for user `785a2703cd6c4ba3a64393098bd99538`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:50.903659 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-5348b4c3-91a5-4979-8340-552f2ef0154d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:50.968936 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:50.972646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1272/2549] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:50 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:50.983710 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e455c-024a-48e9-8af7-93d5dbf9e591 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:50.985598 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3a6e455c-024a-48e9-8af7-93d5dbf9e591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:51.044060 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e455c-024a-48e9-8af7-93d5dbf9e591 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:51.047954 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b4975e-e8a8-491d-87ab-e0bc48421812 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:51.048613 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4975e-e8a8-491d-87ab-e0bc48421812 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:51.048914 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4975e-e8a8-491d-87ab-e0bc48421812 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:51.049187 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4975e-e8a8-491d-87ab-e0bc48421812 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:51.061188 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b4975e-e8a8-491d-87ab-e0bc48421812 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:51.061188 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b4975e-e8a8-491d-87ab-e0bc48421812 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:51.071520 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1278/2550] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:40:52.311718 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348b4c3-91a5-4979-8340-552f2ef0154d None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:52.385744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348b4c3-91a5-4979-8340-552f2ef0154d None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:52.385744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729983a-e739-418a-8e7a-9838a0a91f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:52.385744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729983a-e739-418a-8e7a-9838a0a91f42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:52.385744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729983a-e739-418a-8e7a-9838a0a91f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:52.386295 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3729983a-e739-418a-8e7a-9838a0a91f42 admin admin] PATH_INFO: `/v3/users/f32f66429de34d348bd44aa123230f64` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:52.386295 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729983a-e739-418a-8e7a-9838a0a91f42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f32f66429de34d348bd44aa123230f64)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:52.387442 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3729983a-e739-418a-8e7a-9838a0a91f42 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:52.573686 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-3729983a-e739-418a-8e7a-9838a0a91f42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f32f66429de34d348bd44aa123230f64'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:52.662428 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-3729983a-e739-418a-8e7a-9838a0a91f42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f32f66429de34d348bd44aa123230f64'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:52.675420 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-3729983a-e739-418a-8e7a-9838a0a91f42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f32f66429de34d348bd44aa123230f64'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:52.676075 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-3729983a-e739-418a-8e7a-9838a0a91f42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f32f66429de34d348bd44aa123230f64'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:52.676684 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-3729983a-e739-418a-8e7a-9838a0a91f42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f32f66429de34d348bd44aa123230f64'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:52.689394 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1273/2551] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:40:52 2022] DELETE /identity/v3/users/f32f66429de34d348bd44aa123230f64 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:52.694067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc462a-25a0-49ef-b3a4-de8523c8529f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:52.694535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc462a-25a0-49ef-b3a4-de8523c8529f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:52.694646 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc462a-25a0-49ef-b3a4-de8523c8529f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:52.712867 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-94bc462a-25a0-49ef-b3a4-de8523c8529f None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:52.714065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-94bc462a-25a0-49ef-b3a4-de8523c8529f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:52.773172 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1279/2552] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:52.783365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729983a-e739-418a-8e7a-9838a0a91f42 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:52.785320 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-3729983a-e739-418a-8e7a-9838a0a91f42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:52.848530 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3729983a-e739-418a-8e7a-9838a0a91f42 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:52.852145 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db1a64-20d7-48cf-90f8-bb9803f46757 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:52.852787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db1a64-20d7-48cf-90f8-bb9803f46757 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:52.853069 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db1a64-20d7-48cf-90f8-bb9803f46757 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:52.853343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9db1a64-20d7-48cf-90f8-bb9803f46757 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:52.862738 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db1a64-20d7-48cf-90f8-bb9803f46757 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:52.864202 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9db1a64-20d7-48cf-90f8-bb9803f46757 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:52.873429 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1274/2553] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:40:53.349978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc462a-25a0-49ef-b3a4-de8523c8529f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:53.358720 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-94bc462a-25a0-49ef-b3a4-de8523c8529f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:53.364606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc462a-25a0-49ef-b3a4-de8523c8529f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:53.369676 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc3399-b5b5-417e-bc3f-99e91ee41b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:53.370456 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc3399-b5b5-417e-bc3f-99e91ee41b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:53.370737 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc3399-b5b5-417e-bc3f-99e91ee41b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:53.371014 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc3399-b5b5-417e-bc3f-99e91ee41b9a admin admin] PATH_INFO: `/v3/projects/ad9a9c86adf24a1ab226f770fe6907d9` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:53.372219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc3399-b5b5-417e-bc3f-99e91ee41b9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad9a9c86adf24a1ab226f770fe6907d9)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:53.374143 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc3399-b5b5-417e-bc3f-99e91ee41b9a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:53.476917 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-befc3399-b5b5-417e-bc3f-99e91ee41b9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad9a9c86adf24a1ab226f770fe6907d9', 'request_id': 'req-befc3399-b5b5-417e-bc3f-99e91ee41b9a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:53.537820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-befc3399-b5b5-417e-bc3f-99e91ee41b9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad9a9c86adf24a1ab226f770fe6907d9', 'request_id': 'req-befc3399-b5b5-417e-bc3f-99e91ee41b9a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:53.544173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-befc3399-b5b5-417e-bc3f-99e91ee41b9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad9a9c86adf24a1ab226f770fe6907d9', 'request_id': 'req-befc3399-b5b5-417e-bc3f-99e91ee41b9a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:53.563891 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-befc3399-b5b5-417e-bc3f-99e91ee41b9a admin admin] The token cache is being invalidate because project ad9a9c86adf24a1ab226f770fe6907d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:40:53.563891 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-befc3399-b5b5-417e-bc3f-99e91ee41b9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:53.563891 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-befc3399-b5b5-417e-bc3f-99e91ee41b9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:53.563891 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1280/2554] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:40:53 2022] DELETE /identity/v3/projects/ad9a9c86adf24a1ab226f770fe6907d9 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:53.564279 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db1a64-20d7-48cf-90f8-bb9803f46757 service neutron] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:53.572489 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9db1a64-20d7-48cf-90f8-bb9803f46757 service neutron] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:53.591322 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:53.629360 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f5683-1425-47c8-b1ea-15c021746fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:53.630162 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f5683-1425-47c8-b1ea-15c021746fd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:53.630447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f5683-1425-47c8-b1ea-15c021746fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:53.630725 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f5683-1425-47c8-b1ea-15c021746fd6 admin admin] PATH_INFO: `/v3/users/785a2703cd6c4ba3a64393098bd99538` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:53.631948 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645f5683-1425-47c8-b1ea-15c021746fd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=785a2703cd6c4ba3a64393098bd99538)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:53.633893 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645f5683-1425-47c8-b1ea-15c021746fd6 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:53.788333 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-645f5683-1425-47c8-b1ea-15c021746fd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '785a2703cd6c4ba3a64393098bd99538'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:53.865330 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-645f5683-1425-47c8-b1ea-15c021746fd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '785a2703cd6c4ba3a64393098bd99538'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:53.885775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-645f5683-1425-47c8-b1ea-15c021746fd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '785a2703cd6c4ba3a64393098bd99538'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:53.886400 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-645f5683-1425-47c8-b1ea-15c021746fd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '785a2703cd6c4ba3a64393098bd99538'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:53.887006 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-645f5683-1425-47c8-b1ea-15c021746fd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '785a2703cd6c4ba3a64393098bd99538'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:53.900626 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1275/2555] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:40:53 2022] DELETE /identity/v3/users/785a2703cd6c4ba3a64393098bd99538 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:54.252341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc3399-b5b5-417e-bc3f-99e91ee41b9a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:54.253811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-befc3399-b5b5-417e-bc3f-99e91ee41b9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:54.267124 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc3399-b5b5-417e-bc3f-99e91ee41b9a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:54.275795 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:54.324274 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369c6790-95fc-486b-824e-64119e0a0a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:54.325137 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c6790-95fc-486b-824e-64119e0a0a25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:54.325417 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c6790-95fc-486b-824e-64119e0a0a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:54.325691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369c6790-95fc-486b-824e-64119e0a0a25 admin admin] PATH_INFO: `/v3/users/b9bff5929e654b41aff1d27a99981840` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:54.326858 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369c6790-95fc-486b-824e-64119e0a0a25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9bff5929e654b41aff1d27a99981840)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:54.329276 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369c6790-95fc-486b-824e-64119e0a0a25 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:54.365033 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f5683-1425-47c8-b1ea-15c021746fd6 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:54.365033 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-645f5683-1425-47c8-b1ea-15c021746fd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:54.379487 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f5683-1425-47c8-b1ea-15c021746fd6 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:54.384891 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8edbc1f-7816-448d-9eeb-be85b358e5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:54.385897 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8edbc1f-7816-448d-9eeb-be85b358e5b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:54.386324 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8edbc1f-7816-448d-9eeb-be85b358e5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:54.387149 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8edbc1f-7816-448d-9eeb-be85b358e5b2 admin admin] PATH_INFO: `/v3/projects/1d845010a9fc4ce99410ca30b2436b83` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:54.388954 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8edbc1f-7816-448d-9eeb-be85b358e5b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d845010a9fc4ce99410ca30b2436b83)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:54.390910 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8edbc1f-7816-448d-9eeb-be85b358e5b2 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:54.586670 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e8edbc1f-7816-448d-9eeb-be85b358e5b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d845010a9fc4ce99410ca30b2436b83', 'request_id': 'req-e8edbc1f-7816-448d-9eeb-be85b358e5b2'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:54.588219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-369c6790-95fc-486b-824e-64119e0a0a25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9bff5929e654b41aff1d27a99981840'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:54.588887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-369c6790-95fc-486b-824e-64119e0a0a25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9bff5929e654b41aff1d27a99981840'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:54.594052 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e8edbc1f-7816-448d-9eeb-be85b358e5b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d845010a9fc4ce99410ca30b2436b83', 'request_id': 'req-e8edbc1f-7816-448d-9eeb-be85b358e5b2'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:54.597279 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-369c6790-95fc-486b-824e-64119e0a0a25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9bff5929e654b41aff1d27a99981840'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:54.598524 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-369c6790-95fc-486b-824e-64119e0a0a25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9bff5929e654b41aff1d27a99981840'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:54.602476 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e8edbc1f-7816-448d-9eeb-be85b358e5b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d845010a9fc4ce99410ca30b2436b83', 'request_id': 'req-e8edbc1f-7816-448d-9eeb-be85b358e5b2'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:54.688370 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-369c6790-95fc-486b-824e-64119e0a0a25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9bff5929e654b41aff1d27a99981840'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:54.706929 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1281/2556] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:40:54 2022] DELETE /identity/v3/users/b9bff5929e654b41aff1d27a99981840 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:40:54.711618 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c5c73-5ac0-46c1-8796-b651b9ecc857 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:54.711938 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c5c73-5ac0-46c1-8796-b651b9ecc857 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:54.712221 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c5c73-5ac0-46c1-8796-b651b9ecc857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:54.730623 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-4e1c5c73-5ac0-46c1-8796-b651b9ecc857 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:54.731862 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4e1c5c73-5ac0-46c1-8796-b651b9ecc857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:54.759292 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e8edbc1f-7816-448d-9eeb-be85b358e5b2 admin admin] The token cache is being invalidate because project 1d845010a9fc4ce99410ca30b2436b83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:40:54.759719 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e8edbc1f-7816-448d-9eeb-be85b358e5b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:54.760412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e8edbc1f-7816-448d-9eeb-be85b358e5b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:54.762011 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1276/2557] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:40:54 2022] DELETE /identity/v3/projects/1d845010a9fc4ce99410ca30b2436b83 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:40:54.767546 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1d04e-ebf3-4be0-a62e-9a76405847c1 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:54.767835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1d04e-ebf3-4be0-a62e-9a76405847c1 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:54.768117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1d04e-ebf3-4be0-a62e-9a76405847c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:54.786780 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-91f1d04e-ebf3-4be0-a62e-9a76405847c1 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:54.788045 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-91f1d04e-ebf3-4be0-a62e-9a76405847c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:54.792751 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1282/2558] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:40:54.796460 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1277/2559] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:54.803767 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c5c73-5ac0-46c1-8796-b651b9ecc857 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:54.805442 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4e1c5c73-5ac0-46c1-8796-b651b9ecc857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:54.809931 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac3845-2793-4b24-b5ca-415e00b99574 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:54.810317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac3845-2793-4b24-b5ca-415e00b99574 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:54.810656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac3845-2793-4b24-b5ca-415e00b99574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:54.830965 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-5bac3845-2793-4b24-b5ca-415e00b99574 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:54.832237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-5bac3845-2793-4b24-b5ca-415e00b99574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:54.840381 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1278/2560] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:40:54.845117 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac3845-2793-4b24-b5ca-415e00b99574 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:54.849949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:54.854345 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac3845-2793-4b24-b5ca-415e00b99574 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:54.858903 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097a1e6-53b7-4352-bdec-0a3f58fcc0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:54.859632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097a1e6-53b7-4352-bdec-0a3f58fcc0fa admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:54.860582 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097a1e6-53b7-4352-bdec-0a3f58fcc0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:54.864671 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d097a1e6-53b7-4352-bdec-0a3f58fcc0fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:54.864671 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097a1e6-53b7-4352-bdec-0a3f58fcc0fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:40:54.864671 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097a1e6-53b7-4352-bdec-0a3f58fcc0fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:54.864671 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d097a1e6-53b7-4352-bdec-0a3f58fcc0fa admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:54.866201 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c5c73-5ac0-46c1-8796-b651b9ecc857 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:54.870031 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c20b6-6f15-44ed-a67a-eba8b690269c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:54.870900 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c20b6-6f15-44ed-a67a-eba8b690269c service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:54.871042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c20b6-6f15-44ed-a67a-eba8b690269c service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:54.871283 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67c20b6-6f15-44ed-a67a-eba8b690269c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:54.871566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:54.876754 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1279/2561] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:40:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:40:54.881041 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097a1e6-53b7-4352-bdec-0a3f58fcc0fa admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:54.884242 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c20b6-6f15-44ed-a67a-eba8b690269c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:54.885695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67c20b6-6f15-44ed-a67a-eba8b690269c service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:54.891799 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d097a1e6-53b7-4352-bdec-0a3f58fcc0fa admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:54.896294 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2283b67f-331a-4f54-8b95-ad8c7c999f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:54.896938 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283b67f-331a-4f54-8b95-ad8c7c999f57 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:54.897069 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1283/2562] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:40:54.897219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283b67f-331a-4f54-8b95-ad8c7c999f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:54.897502 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2283b67f-331a-4f54-8b95-ad8c7c999f57 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:54.898443 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2283b67f-331a-4f54-8b95-ad8c7c999f57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:54.900416 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2283b67f-331a-4f54-8b95-ad8c7c999f57 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:54.984173 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1280/2563] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:40:54 2022] POST /identity/v3/projects => generated 361 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:40:54.988864 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c20b6-6f15-44ed-a67a-eba8b690269c service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:54.998547 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67c20b6-6f15-44ed-a67a-eba8b690269c service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:55.053991 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f8f04-a803-4cd6-82a7-e6db53d8e0fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:55.054664 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f8f04-a803-4cd6-82a7-e6db53d8e0fe admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:55.054954 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f8f04-a803-4cd6-82a7-e6db53d8e0fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:55.055235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587f8f04-a803-4cd6-82a7-e6db53d8e0fe admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:55.055966 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f8f04-a803-4cd6-82a7-e6db53d8e0fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:55.057890 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587f8f04-a803-4cd6-82a7-e6db53d8e0fe admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:55.220700 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1284/2564] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:40:54 2022] POST /identity/v3/users => generated 281 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:40:55.224908 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2283b67f-331a-4f54-8b95-ad8c7c999f57 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:55.235832 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2283b67f-331a-4f54-8b95-ad8c7c999f57 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:55.240314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8b1d0-3b29-49e6-ace5-e43fbc2af351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:55.240981 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8b1d0-3b29-49e6-ace5-e43fbc2af351 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:55.241294 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8b1d0-3b29-49e6-ace5-e43fbc2af351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:55.241581 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd8b1d0-3b29-49e6-ace5-e43fbc2af351 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:55.242331 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd8b1d0-3b29-49e6-ace5-e43fbc2af351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:55.243941 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd8b1d0-3b29-49e6-ace5-e43fbc2af351 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:55.254318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1281/2565] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:40:55.265359 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f8f04-a803-4cd6-82a7-e6db53d8e0fe admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:55.269578 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587f8f04-a803-4cd6-82a7-e6db53d8e0fe admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:55.274169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6fa85-3783-454b-98f6-308851d96ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:55.274820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6fa85-3783-454b-98f6-308851d96ade admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:55.275103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6fa85-3783-454b-98f6-308851d96ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:55.275433 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6fa85-3783-454b-98f6-308851d96ade admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:55.276193 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6fa85-3783-454b-98f6-308851d96ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:55.277799 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6fa85-3783-454b-98f6-308851d96ade admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:55.288908 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1285/2566] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:40:55.297415 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8b1d0-3b29-49e6-ace5-e43fbc2af351 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:55.307285 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd8b1d0-3b29-49e6-ace5-e43fbc2af351 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:55.311751 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce114980-72ae-45d8-8b24-98c4afec8f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:55.312624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce114980-72ae-45d8-8b24-98c4afec8f23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:55.313487 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce114980-72ae-45d8-8b24-98c4afec8f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:55.313774 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce114980-72ae-45d8-8b24-98c4afec8f23 admin admin] PATH_INFO: `/v3/projects/1fe49195931245bda34fcb4f174726d7/users/f1923bfab2aa4a8ea9d54a7a8a558177/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:55.328340 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce114980-72ae-45d8-8b24-98c4afec8f23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fe49195931245bda34fcb4f174726d7, user_id=f1923bfab2aa4a8ea9d54a7a8a558177, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:55.331674 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce114980-72ae-45d8-8b24-98c4afec8f23 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:55.355133 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6fa85-3783-454b-98f6-308851d96ade admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:55.356955 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-69c6fa85-3783-454b-98f6-308851d96ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:55.370830 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6fa85-3783-454b-98f6-308851d96ade admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:55.376452 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca79748-debc-4017-9e11-7b68dca8d6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:55.377320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca79748-debc-4017-9e11-7b68dca8d6de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:55.377612 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca79748-debc-4017-9e11-7b68dca8d6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:55.377892 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca79748-debc-4017-9e11-7b68dca8d6de admin admin] PATH_INFO: `/v3/projects/f4c9c588fdb54c10be72ad24b8465995` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:55.379108 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca79748-debc-4017-9e11-7b68dca8d6de admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4c9c588fdb54c10be72ad24b8465995)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:55.381071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca79748-debc-4017-9e11-7b68dca8d6de admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:55.428641 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1282/2567] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:40:55 2022] PUT /identity/v3/projects/1fe49195931245bda34fcb4f174726d7/users/f1923bfab2aa4a8ea9d54a7a8a558177/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:40:55.440807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2493a-1752-4454-b94d-bfd0d862b25a None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:55.441096 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2493a-1752-4454-b94d-bfd0d862b25a None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:55.441371 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a2493a-1752-4454-b94d-bfd0d862b25a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:55.468827 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-79a2493a-1752-4454-b94d-bfd0d862b25a None None] MFA Rules not processed for user `f1923bfab2aa4a8ea9d54a7a8a558177`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:55.470361 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-79a2493a-1752-4454-b94d-bfd0d862b25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:55.517693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2ca79748-debc-4017-9e11-7b68dca8d6de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4c9c588fdb54c10be72ad24b8465995', 'request_id': 'req-2ca79748-debc-4017-9e11-7b68dca8d6de'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:55.540884 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1283/2568] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:40:55.576627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2493a-1752-4454-b94d-bfd0d862b25a None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:55.577999 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-79a2493a-1752-4454-b94d-bfd0d862b25a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:55.614084 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2ca79748-debc-4017-9e11-7b68dca8d6de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4c9c588fdb54c10be72ad24b8465995', 'request_id': 'req-2ca79748-debc-4017-9e11-7b68dca8d6de'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:55.620500 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2ca79748-debc-4017-9e11-7b68dca8d6de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4c9c588fdb54c10be72ad24b8465995', 'request_id': 'req-2ca79748-debc-4017-9e11-7b68dca8d6de'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:55.633366 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2ca79748-debc-4017-9e11-7b68dca8d6de admin admin] The token cache is being invalidate because project f4c9c588fdb54c10be72ad24b8465995 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:40:55.633651 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2ca79748-debc-4017-9e11-7b68dca8d6de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:55.634380 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2ca79748-debc-4017-9e11-7b68dca8d6de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:55.635040 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a2493a-1752-4454-b94d-bfd0d862b25a None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:55.636297 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1286/2569] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:40:55 2022] DELETE /identity/v3/projects/f4c9c588fdb54c10be72ad24b8465995 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:40:55.642774 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381033d-0307-4d54-a4a0-28742674d66f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:55.643060 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381033d-0307-4d54-a4a0-28742674d66f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:55.643360 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9381033d-0307-4d54-a4a0-28742674d66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:55.661368 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9381033d-0307-4d54-a4a0-28742674d66f None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:55.662600 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9381033d-0307-4d54-a4a0-28742674d66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:55.679829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:55.688755 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b38e8-94bd-48cb-953f-39ab58598c73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:55.688755 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b38e8-94bd-48cb-953f-39ab58598c73 service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:55.688755 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b38e8-94bd-48cb-953f-39ab58598c73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:55.688755 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b38e8-94bd-48cb-953f-39ab58598c73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:55.689147 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-cd7b38e8-94bd-48cb-953f-39ab58598c73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:55.700172 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:55.724716 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1287/2570] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:40:55.733010 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c41a1-44b5-4752-8456-88d3114b447a None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:55.733297 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c41a1-44b5-4752-8456-88d3114b447a None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:55.733573 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06c41a1-44b5-4752-8456-88d3114b447a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:55.747717 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b38e8-94bd-48cb-953f-39ab58598c73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:55.749233 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b38e8-94bd-48cb-953f-39ab58598c73 service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:55.752665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f06c41a1-44b5-4752-8456-88d3114b447a None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:55.756304 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f06c41a1-44b5-4752-8456-88d3114b447a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:55.761893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1288/2572] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:40:55.765868 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c41a1-44b5-4752-8456-88d3114b447a None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:55.767337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1284/2571] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:55 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:40:55.774192 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b38e8-94bd-48cb-953f-39ab58598c73 service cinder] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:55.775729 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-cd7b38e8-94bd-48cb-953f-39ab58598c73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:55.776125 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06c41a1-44b5-4752-8456-88d3114b447a None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:55.781857 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c303a4-a961-4d2c-a283-ed7ec4ea9668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:55.783000 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c303a4-a961-4d2c-a283-ed7ec4ea9668 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:55.783304 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c303a4-a961-4d2c-a283-ed7ec4ea9668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:55.790412 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c303a4-a961-4d2c-a283-ed7ec4ea9668 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:55.790412 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c303a4-a961-4d2c-a283-ed7ec4ea9668 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:40:55.790412 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c303a4-a961-4d2c-a283-ed7ec4ea9668 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:55.790412 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c303a4-a961-4d2c-a283-ed7ec4ea9668 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:55.790802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b38e8-94bd-48cb-953f-39ab58598c73 service cinder] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:55.794717 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde0f45-fdd6-4ed2-83f3-9d38cacb3425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:55.795664 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde0f45-fdd6-4ed2-83f3-9d38cacb3425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:55.795946 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde0f45-fdd6-4ed2-83f3-9d38cacb3425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:55.796250 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde0f45-fdd6-4ed2-83f3-9d38cacb3425 admin admin] PATH_INFO: `/v3/users/f1923bfab2aa4a8ea9d54a7a8a558177` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:55.798547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde0f45-fdd6-4ed2-83f3-9d38cacb3425 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1923bfab2aa4a8ea9d54a7a8a558177)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:55.800666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde0f45-fdd6-4ed2-83f3-9d38cacb3425 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:55.802848 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1289/2573] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:40:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:40:55.807305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c303a4-a961-4d2c-a283-ed7ec4ea9668 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:55.818583 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c303a4-a961-4d2c-a283-ed7ec4ea9668 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:55.824129 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a507670-2eb8-40c2-b8db-1001bb82e6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:55.824946 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a507670-2eb8-40c2-b8db-1001bb82e6e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:55.825465 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a507670-2eb8-40c2-b8db-1001bb82e6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:55.825814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a507670-2eb8-40c2-b8db-1001bb82e6e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:55.826668 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a507670-2eb8-40c2-b8db-1001bb82e6e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:55.828881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a507670-2eb8-40c2-b8db-1001bb82e6e3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:56.053676 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1290/2574] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:40:55 2022] POST /identity/v3/projects => generated 401 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:40:56.057921 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a507670-2eb8-40c2-b8db-1001bb82e6e3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:56.119111 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a507670-2eb8-40c2-b8db-1001bb82e6e3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:56.123889 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c7dd6-af40-412d-bef8-39c7f2acc66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:56.124580 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c7dd6-af40-412d-bef8-39c7f2acc66e admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:56.124868 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c7dd6-af40-412d-bef8-39c7f2acc66e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:56.125148 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c7dd6-af40-412d-bef8-39c7f2acc66e admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:56.125868 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c7dd6-af40-412d-bef8-39c7f2acc66e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:56.127844 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c7dd6-af40-412d-bef8-39c7f2acc66e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:56.151794 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-6bde0f45-fdd6-4ed2-83f3-9d38cacb3425 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1923bfab2aa4a8ea9d54a7a8a558177'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:56.249298 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1291/2575] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:40:56 2022] POST /identity/v3/users => generated 301 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:40:56.253579 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c7dd6-af40-412d-bef8-39c7f2acc66e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:56.263242 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c7dd6-af40-412d-bef8-39c7f2acc66e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:56.267870 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aace812-12fd-4ca5-95cd-97aad27eb340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:56.268512 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aace812-12fd-4ca5-95cd-97aad27eb340 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:56.268797 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aace812-12fd-4ca5-95cd-97aad27eb340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:56.269072 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aace812-12fd-4ca5-95cd-97aad27eb340 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:56.269847 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aace812-12fd-4ca5-95cd-97aad27eb340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:56.271476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aace812-12fd-4ca5-95cd-97aad27eb340 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:56.279126 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-6bde0f45-fdd6-4ed2-83f3-9d38cacb3425 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1923bfab2aa4a8ea9d54a7a8a558177'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:56.286493 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1292/2576] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:40:56.287690 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aace812-12fd-4ca5-95cd-97aad27eb340 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:56.293376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-6bde0f45-fdd6-4ed2-83f3-9d38cacb3425 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1923bfab2aa4a8ea9d54a7a8a558177'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:56.294069 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-6bde0f45-fdd6-4ed2-83f3-9d38cacb3425 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1923bfab2aa4a8ea9d54a7a8a558177'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:56.294704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-6bde0f45-fdd6-4ed2-83f3-9d38cacb3425 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1923bfab2aa4a8ea9d54a7a8a558177'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:56.298540 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aace812-12fd-4ca5-95cd-97aad27eb340 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:56.304422 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd9d4dc-e490-4485-870a-8eb94da7d15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:56.305060 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9d4dc-e490-4485-870a-8eb94da7d15d admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:56.305351 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9d4dc-e490-4485-870a-8eb94da7d15d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:56.305630 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd9d4dc-e490-4485-870a-8eb94da7d15d admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:56.306355 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd9d4dc-e490-4485-870a-8eb94da7d15d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:56.307997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd9d4dc-e490-4485-870a-8eb94da7d15d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:56.308434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1285/2577] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:40:55 2022] DELETE /identity/v3/users/f1923bfab2aa4a8ea9d54a7a8a558177 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:56.313259 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2feaab0-ebff-4ab4-bf32-1e88adeabf61 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:56.313665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2feaab0-ebff-4ab4-bf32-1e88adeabf61 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:56.321446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2feaab0-ebff-4ab4-bf32-1e88adeabf61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:56.321563 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1293/2578] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:40:56.324254 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd9d4dc-e490-4485-870a-8eb94da7d15d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:56.326402 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7bd9d4dc-e490-4485-870a-8eb94da7d15d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:56.331112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-b2feaab0-ebff-4ab4-bf32-1e88adeabf61 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:56.332410 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-b2feaab0-ebff-4ab4-bf32-1e88adeabf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:56.338963 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd9d4dc-e490-4485-870a-8eb94da7d15d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:56.394846 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1286/2579] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:56.400723 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f97e2e-1571-4480-8aa7-e53f5809cbc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:56.401740 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f97e2e-1571-4480-8aa7-e53f5809cbc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:56.402060 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f97e2e-1571-4480-8aa7-e53f5809cbc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:56.402406 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f97e2e-1571-4480-8aa7-e53f5809cbc9 admin admin] PATH_INFO: `/v3/projects/37e1fb79754c485f94d8a0c814760cf1/users/2741c615871c4ac59304338e4e746b60/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:56.404912 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2feaab0-ebff-4ab4-bf32-1e88adeabf61 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:56.406371 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-b2feaab0-ebff-4ab4-bf32-1e88adeabf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:56.413767 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:56.420939 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f97e2e-1571-4480-8aa7-e53f5809cbc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37e1fb79754c485f94d8a0c814760cf1, user_id=2741c615871c4ac59304338e4e746b60, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:56.421490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:56.423674 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f97e2e-1571-4480-8aa7-e53f5809cbc9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:56.464845 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2feaab0-ebff-4ab4-bf32-1e88adeabf61 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:56.468620 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9b19bb-196c-4036-8bee-c6bde8d6f323 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:56.469991 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b19bb-196c-4036-8bee-c6bde8d6f323 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:56.470291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b19bb-196c-4036-8bee-c6bde8d6f323 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:56.470576 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9b19bb-196c-4036-8bee-c6bde8d6f323 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:56.479554 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b19bb-196c-4036-8bee-c6bde8d6f323 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:56.480973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9b19bb-196c-4036-8bee-c6bde8d6f323 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:56.491481 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1287/2580] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:40:56.491776 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1294/2581] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:40:56 2022] PUT /identity/v3/projects/37e1fb79754c485f94d8a0c814760cf1/users/2741c615871c4ac59304338e4e746b60/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:40:56.503300 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c84b8-0fdb-476c-8da3-ad909d9a4516 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:56.503696 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c84b8-0fdb-476c-8da3-ad909d9a4516 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:56.504039 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100c84b8-0fdb-476c-8da3-ad909d9a4516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:56.531496 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-100c84b8-0fdb-476c-8da3-ad909d9a4516 None None] MFA Rules not processed for user `2741c615871c4ac59304338e4e746b60`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:56.533498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-100c84b8-0fdb-476c-8da3-ad909d9a4516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:56.598570 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1288/2582] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:56 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:56.603195 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f97e2e-1571-4480-8aa7-e53f5809cbc9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:56.664818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f97e2e-1571-4480-8aa7-e53f5809cbc9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:56.669425 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9095241-ab53-4699-bab0-050ee564bfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:56.670144 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9095241-ab53-4699-bab0-050ee564bfe3 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:56.670436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9095241-ab53-4699-bab0-050ee564bfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:56.670706 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9095241-ab53-4699-bab0-050ee564bfe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:56.671417 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9095241-ab53-4699-bab0-050ee564bfe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:56.673341 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9095241-ab53-4699-bab0-050ee564bfe3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:56.756542 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1295/2583] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:40:56 2022] POST /identity/v3/projects => generated 401 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:40:56.760924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c84b8-0fdb-476c-8da3-ad909d9a4516 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:56.772859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100c84b8-0fdb-476c-8da3-ad909d9a4516 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:56.833274 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0e82c-de79-42e9-8a5a-ced03a72e9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:56.834007 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0e82c-de79-42e9-8a5a-ced03a72e9ac admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:56.834288 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0e82c-de79-42e9-8a5a-ced03a72e9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:56.834562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f0e82c-de79-42e9-8a5a-ced03a72e9ac admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:56.835341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f0e82c-de79-42e9-8a5a-ced03a72e9ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:56.837299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f0e82c-de79-42e9-8a5a-ced03a72e9ac admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:56.978532 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1289/2584] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:40:56 2022] POST /identity/v3/users => generated 307 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:40:56.980209 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9095241-ab53-4699-bab0-050ee564bfe3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:56.982304 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e9095241-ab53-4699-bab0-050ee564bfe3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:56.986840 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0e82c-de79-42e9-8a5a-ced03a72e9ac admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:57.003663 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9095241-ab53-4699-bab0-050ee564bfe3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:57.008986 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f0e82c-de79-42e9-8a5a-ced03a72e9ac admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:57.015763 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439f7f91-43bd-45d7-bc3b-8678fa0e4dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:57.017318 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f7f91-43bd-45d7-bc3b-8678fa0e4dbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:57.017605 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f7f91-43bd-45d7-bc3b-8678fa0e4dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:57.017868 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439f7f91-43bd-45d7-bc3b-8678fa0e4dbf admin admin] PATH_INFO: `/v3/projects/1fe49195931245bda34fcb4f174726d7` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:57.019236 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439f7f91-43bd-45d7-bc3b-8678fa0e4dbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fe49195931245bda34fcb4f174726d7)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:57.020189 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2744e-5e84-4bf2-a0e7-a82f6291c86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:57.021017 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2744e-5e84-4bf2-a0e7-a82f6291c86f admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:57.021331 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439f7f91-43bd-45d7-bc3b-8678fa0e4dbf admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:57.021810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2744e-5e84-4bf2-a0e7-a82f6291c86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:57.022189 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d2744e-5e84-4bf2-a0e7-a82f6291c86f admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:57.023036 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2744e-5e84-4bf2-a0e7-a82f6291c86f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:57.024698 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d2744e-5e84-4bf2-a0e7-a82f6291c86f admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:57.036243 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1290/2585] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:56 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:40:57.040002 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2744e-5e84-4bf2-a0e7-a82f6291c86f admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:57.050738 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d2744e-5e84-4bf2-a0e7-a82f6291c86f admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:57.056388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d5068-925a-4eff-ac16-75ced3454021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:57.057523 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d5068-925a-4eff-ac16-75ced3454021 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:57.057877 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d5068-925a-4eff-ac16-75ced3454021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:57.058311 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d5068-925a-4eff-ac16-75ced3454021 admin admin] PATH_INFO: `/v3/projects/eb03759583a546f48e954ac025ef935d/users/b647b2790d27430fae9f55e04ec7a847/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:57.075129 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177d5068-925a-4eff-ac16-75ced3454021 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb03759583a546f48e954ac025ef935d, user_id=b647b2790d27430fae9f55e04ec7a847, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:57.077895 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177d5068-925a-4eff-ac16-75ced3454021 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:57.161612 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1291/2586] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:40:57 2022] PUT /identity/v3/projects/eb03759583a546f48e954ac025ef935d/users/b647b2790d27430fae9f55e04ec7a847/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:40:57.165542 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d5068-925a-4eff-ac16-75ced3454021 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:57.169433 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-439f7f91-43bd-45d7-bc3b-8678fa0e4dbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fe49195931245bda34fcb4f174726d7', 'request_id': 'req-439f7f91-43bd-45d7-bc3b-8678fa0e4dbf'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:57.227335 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d5068-925a-4eff-ac16-75ced3454021 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:57.284933 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ac0a2-8a7e-494e-9c2a-25576507fc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:57.285619 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ac0a2-8a7e-494e-9c2a-25576507fc13 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:57.286161 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ac0a2-8a7e-494e-9c2a-25576507fc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:57.286431 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676ac0a2-8a7e-494e-9c2a-25576507fc13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:57.287232 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ac0a2-8a7e-494e-9c2a-25576507fc13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:57.288741 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-439f7f91-43bd-45d7-bc3b-8678fa0e4dbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fe49195931245bda34fcb4f174726d7', 'request_id': 'req-439f7f91-43bd-45d7-bc3b-8678fa0e4dbf'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:57.289269 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676ac0a2-8a7e-494e-9c2a-25576507fc13 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:57.296153 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-439f7f91-43bd-45d7-bc3b-8678fa0e4dbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fe49195931245bda34fcb4f174726d7', 'request_id': 'req-439f7f91-43bd-45d7-bc3b-8678fa0e4dbf'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:57.301368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1292/2587] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:40:57.305808 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ac0a2-8a7e-494e-9c2a-25576507fc13 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:57.314354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-439f7f91-43bd-45d7-bc3b-8678fa0e4dbf admin admin] The token cache is being invalidate because project 1fe49195931245bda34fcb4f174726d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:40:57.314647 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-439f7f91-43bd-45d7-bc3b-8678fa0e4dbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:57.315597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-439f7f91-43bd-45d7-bc3b-8678fa0e4dbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:40:57.317173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1296/2588] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:40:56 2022] DELETE /identity/v3/projects/1fe49195931245bda34fcb4f174726d7 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:40:57.319005 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676ac0a2-8a7e-494e-9c2a-25576507fc13 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:57.321819 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab1c0a-5a29-4721-b01e-36a7143924c9 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:57.322175 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab1c0a-5a29-4721-b01e-36a7143924c9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:57.322523 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beab1c0a-5a29-4721-b01e-36a7143924c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:57.324069 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf625eac-1ca9-4341-9f00-f8ddbe65a987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:57.324959 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf625eac-1ca9-4341-9f00-f8ddbe65a987 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:57.325248 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf625eac-1ca9-4341-9f00-f8ddbe65a987 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:57.325529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf625eac-1ca9-4341-9f00-f8ddbe65a987 admin admin] PATH_INFO: `/v3/projects/eb03759583a546f48e954ac025ef935d/users/b647b2790d27430fae9f55e04ec7a847/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:57.327596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf625eac-1ca9-4341-9f00-f8ddbe65a987 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb03759583a546f48e954ac025ef935d, user_id=b647b2790d27430fae9f55e04ec7a847, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:57.330422 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf625eac-1ca9-4341-9f00-f8ddbe65a987 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:57.340972 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1293/2589] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:40:57 2022] PUT /identity/v3/projects/eb03759583a546f48e954ac025ef935d/users/b647b2790d27430fae9f55e04ec7a847/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:40:57.346461 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-beab1c0a-5a29-4721-b01e-36a7143924c9 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:57.348211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-beab1c0a-5a29-4721-b01e-36a7143924c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:57.352486 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ab6b5-b7b3-4517-862d-70388fa25078 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:57.352773 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ab6b5-b7b3-4517-862d-70388fa25078 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:57.353058 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ab6b5-b7b3-4517-862d-70388fa25078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:57.381280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-d62ab6b5-b7b3-4517-862d-70388fa25078 None None] MFA Rules not processed for user `b647b2790d27430fae9f55e04ec7a847`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:57.382513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d62ab6b5-b7b3-4517-862d-70388fa25078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:57.388311 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:57.404640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:57.416488 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1297/2590] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:40:57.424562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81e729-f8b8-4212-86cb-a5fed5ebac23 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:57.425305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81e729-f8b8-4212-86cb-a5fed5ebac23 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:57.425597 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81e729-f8b8-4212-86cb-a5fed5ebac23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:57.445528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0e81e729-f8b8-4212-86cb-a5fed5ebac23 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:57.446748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0e81e729-f8b8-4212-86cb-a5fed5ebac23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:57.454873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1298/2591] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:40:57.455281 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1294/2592] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:57 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:40:57.464097 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81e729-f8b8-4212-86cb-a5fed5ebac23 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:57.472419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ab6b5-b7b3-4517-862d-70388fa25078 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:57.473654 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81e729-f8b8-4212-86cb-a5fed5ebac23 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:57.473800 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d62ab6b5-b7b3-4517-862d-70388fa25078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:57.478643 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce558719-735e-4871-aa7d-9ed4fb0e008b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:57.479640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce558719-735e-4871-aa7d-9ed4fb0e008b admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:57.480447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce558719-735e-4871-aa7d-9ed4fb0e008b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:57.480732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce558719-735e-4871-aa7d-9ed4fb0e008b admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:57.481464 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce558719-735e-4871-aa7d-9ed4fb0e008b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:40:57.481800 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce558719-735e-4871-aa7d-9ed4fb0e008b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:57.483443 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce558719-735e-4871-aa7d-9ed4fb0e008b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:57.495919 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1299/2593] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:40:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:40:57.500217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce558719-735e-4871-aa7d-9ed4fb0e008b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:57.511664 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce558719-735e-4871-aa7d-9ed4fb0e008b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:57.516609 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7de01-d874-4b12-ae19-e0fadce764b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:57.517273 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7de01-d874-4b12-ae19-e0fadce764b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:57.517687 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7de01-d874-4b12-ae19-e0fadce764b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:57.518012 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7de01-d874-4b12-ae19-e0fadce764b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:57.518755 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7de01-d874-4b12-ae19-e0fadce764b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:57.520759 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc7de01-d874-4b12-ae19-e0fadce764b8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:57.532526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62ab6b5-b7b3-4517-862d-70388fa25078 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:57.536434 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea2a32-4c14-429a-9507-74c54ae18e9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:57.537082 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea2a32-4c14-429a-9507-74c54ae18e9a service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:57.537370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea2a32-4c14-429a-9507-74c54ae18e9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:57.537651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ea2a32-4c14-429a-9507-74c54ae18e9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:57.546978 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ea2a32-4c14-429a-9507-74c54ae18e9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:57.549148 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ea2a32-4c14-429a-9507-74c54ae18e9a service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:57.558559 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1295/2594] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:57 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:40:57.606296 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1300/2595] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:40:57 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:40:57.611166 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea2a32-4c14-429a-9507-74c54ae18e9a service cinder] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:57.621431 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ea2a32-4c14-429a-9507-74c54ae18e9a service cinder] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:57.680471 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c26b6-a5d6-4bba-a4b4-e727a64e3147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:57.680995 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c26b6-a5d6-4bba-a4b4-e727a64e3147 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:57.681224 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c26b6-a5d6-4bba-a4b4-e727a64e3147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:57.681509 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c26b6-a5d6-4bba-a4b4-e727a64e3147 admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:57.682205 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c26b6-a5d6-4bba-a4b4-e727a64e3147 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:57.684331 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c26b6-a5d6-4bba-a4b4-e727a64e3147 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:57.784700 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1296/2596] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:40:57 2022] POST /identity/v3/users => generated 286 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:40:57.788951 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7de01-d874-4b12-ae19-e0fadce764b8 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:57.800271 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7de01-d874-4b12-ae19-e0fadce764b8 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:57.810690 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a683906-6dbd-4d74-a9ed-7c86eab6ae99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:57.811381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a683906-6dbd-4d74-a9ed-7c86eab6ae99 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:57.811609 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a683906-6dbd-4d74-a9ed-7c86eab6ae99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:57.811864 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a683906-6dbd-4d74-a9ed-7c86eab6ae99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:57.812594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a683906-6dbd-4d74-a9ed-7c86eab6ae99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:57.814813 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a683906-6dbd-4d74-a9ed-7c86eab6ae99 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:57.816332 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c26b6-a5d6-4bba-a4b4-e727a64e3147 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:57.828571 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1301/2597] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:40:57.830033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a683906-6dbd-4d74-a9ed-7c86eab6ae99 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:57.839640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a683906-6dbd-4d74-a9ed-7c86eab6ae99 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:57.847712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa738cb-d883-4c87-b826-5345f9283a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:57.848385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa738cb-d883-4c87-b826-5345f9283a6e admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:57.854890 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa738cb-d883-4c87-b826-5345f9283a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:57.854890 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa738cb-d883-4c87-b826-5345f9283a6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:57.854890 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa738cb-d883-4c87-b826-5345f9283a6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:57.854890 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa738cb-d883-4c87-b826-5345f9283a6e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:57.868520 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1302/2598] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:40:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:40:57.869772 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa738cb-d883-4c87-b826-5345f9283a6e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:57.873042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c26b6-a5d6-4bba-a4b4-e727a64e3147 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:57.878338 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:57.880407 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa738cb-d883-4c87-b826-5345f9283a6e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:57.887488 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e5480-4951-4bc2-a239-8b796d2f0d1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:57.887488 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e5480-4951-4bc2-a239-8b796d2f0d1f service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:57.887488 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e5480-4951-4bc2-a239-8b796d2f0d1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:57.887488 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e5480-4951-4bc2-a239-8b796d2f0d1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:57.887837 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-648e5480-4951-4bc2-a239-8b796d2f0d1f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:57.887924 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e2118-c9ce-472a-93a7-f0a4967e622a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:57.887924 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e2118-c9ce-472a-93a7-f0a4967e622a admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:57.887924 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e2118-c9ce-472a-93a7-f0a4967e622a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:57.887924 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e2118-c9ce-472a-93a7-f0a4967e622a admin admin] PATH_INFO: `/v3/projects/5c62eb86ac194a7fa67c670fd6239743/users/e9cf01f5974c45c6bd19b3e51b23d89c/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:57.903322 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e2118-c9ce-472a-93a7-f0a4967e622a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c62eb86ac194a7fa67c670fd6239743, user_id=e9cf01f5974c45c6bd19b3e51b23d89c, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:57.905997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e2118-c9ce-472a-93a7-f0a4967e622a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:57.944650 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648e5480-4951-4bc2-a239-8b796d2f0d1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:57.946099 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648e5480-4951-4bc2-a239-8b796d2f0d1f service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:57.955634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1297/2599] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:57 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:40:58.018739 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1303/2600] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:40:57 2022] PUT /identity/v3/projects/5c62eb86ac194a7fa67c670fd6239743/users/e9cf01f5974c45c6bd19b3e51b23d89c/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:40:58.024251 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce3569-bf26-48bd-9e9d-84c8875caba7 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:58.024548 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce3569-bf26-48bd-9e9d-84c8875caba7 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:58.024830 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce3569-bf26-48bd-9e9d-84c8875caba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:58.044660 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-e3ce3569-bf26-48bd-9e9d-84c8875caba7 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:58.046108 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-e3ce3569-bf26-48bd-9e9d-84c8875caba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:58.108050 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1298/2601] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:40:58.123071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e2118-c9ce-472a-93a7-f0a4967e622a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:58.123071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-422e2118-c9ce-472a-93a7-f0a4967e622a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:58.178606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e2118-c9ce-472a-93a7-f0a4967e622a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:58.182709 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4608bc6-60a9-4998-90c3-805e9e076fb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:58.183397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4608bc6-60a9-4998-90c3-805e9e076fb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:58.183685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4608bc6-60a9-4998-90c3-805e9e076fb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:58.183967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4608bc6-60a9-4998-90c3-805e9e076fb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:58.194071 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4608bc6-60a9-4998-90c3-805e9e076fb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:58.195562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4608bc6-60a9-4998-90c3-805e9e076fb9 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:58.206227 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1304/2602] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:40:58.371462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:40:59.562996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7db20d-86f5-4467-a159-ee3ce19726a2 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:59.564076 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7db20d-86f5-4467-a159-ee3ce19726a2 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:59.564076 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7db20d-86f5-4467-a159-ee3ce19726a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:59.592362 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-2c7db20d-86f5-4467-a159-ee3ce19726a2 None None] MFA Rules not processed for user `e9cf01f5974c45c6bd19b3e51b23d89c`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:40:59.593588 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-2c7db20d-86f5-4467-a159-ee3ce19726a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:59.680813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1299/2603] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:40:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:40:59.693606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4608bc6-60a9-4998-90c3-805e9e076fb9 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:40:59.697593 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b4608bc6-60a9-4998-90c3-805e9e076fb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:40:59.766568 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4608bc6-60a9-4998-90c3-805e9e076fb9 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:40:59.771684 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e01c8-f4b6-421a-a82e-15a31ac2feee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:40:59.772618 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e01c8-f4b6-421a-a82e-15a31ac2feee service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:40:59.773121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e01c8-f4b6-421a-a82e-15a31ac2feee service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:40:59.773635 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e01c8-f4b6-421a-a82e-15a31ac2feee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:40:59.784604 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e01c8-f4b6-421a-a82e-15a31ac2feee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:40:59.786332 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e01c8-f4b6-421a-a82e-15a31ac2feee service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:40:59.797783 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1305/2604] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:40:59 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:41:01.177387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f248f4-9482-4d46-94c3-a76ec53f9e07 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:01.177891 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f248f4-9482-4d46-94c3-a76ec53f9e07 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:01.178002 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f248f4-9482-4d46-94c3-a76ec53f9e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:01.195796 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-c4f248f4-9482-4d46-94c3-a76ec53f9e07 None None] MFA Rules not processed for user `66ad8defb9b6453c96f1a8a7343695ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:41:01.197040 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-c4f248f4-9482-4d46-94c3-a76ec53f9e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:01.254175 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1300/2605] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jun 30 10:41:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:41:01.484727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e01c8-f4b6-421a-a82e-15a31ac2feee service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:01.486618 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6f1e01c8-f4b6-421a-a82e-15a31ac2feee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:01.507881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:41:01.562006 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e01c8-f4b6-421a-a82e-15a31ac2feee service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:01.568138 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8e5b3-0491-4e62-9632-7016c7ba086d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07de926cb8594b098f79111dab8a9cc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:01.568859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8e5b3-0491-4e62-9632-7016c7ba086d service swift] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:01.569140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8e5b3-0491-4e62-9632-7016c7ba086d service swift] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:01.569524 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe8e5b3-0491-4e62-9632-7016c7ba086d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:01.580103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8e5b3-0491-4e62-9632-7016c7ba086d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:01.581588 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe8e5b3-0491-4e62-9632-7016c7ba086d service swift] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:01.594447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1306/2606] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jun 30 10:41:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:41:01.819940 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77942984-f2f6-4fa7-af7c-15f9c0c5504d None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:01.820441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77942984-f2f6-4fa7-af7c-15f9c0c5504d None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:01.820973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77942984-f2f6-4fa7-af7c-15f9c0c5504d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:01.843251 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-77942984-f2f6-4fa7-af7c-15f9c0c5504d None None] MFA Rules not processed for user `e9cf01f5974c45c6bd19b3e51b23d89c`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:41:01.845773 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-77942984-f2f6-4fa7-af7c-15f9c0c5504d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:01.854299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1301/2607] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:41:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:41:03.476842 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8e5b3-0491-4e62-9632-7016c7ba086d service swift] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:03.478480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4fe8e5b3-0491-4e62-9632-7016c7ba086d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:03.495175 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe8e5b3-0491-4e62-9632-7016c7ba086d service swift] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:03.505290 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5235fa1-69c1-469b-88c1-d42d0b176666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:03.506194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5235fa1-69c1-469b-88c1-d42d0b176666 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:03.506487 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5235fa1-69c1-469b-88c1-d42d0b176666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:03.506770 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5235fa1-69c1-469b-88c1-d42d0b176666 admin admin] PATH_INFO: `/v3/users/2741c615871c4ac59304338e4e746b60` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:03.508044 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5235fa1-69c1-469b-88c1-d42d0b176666 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2741c615871c4ac59304338e4e746b60)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:03.510214 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5235fa1-69c1-469b-88c1-d42d0b176666 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:03.808699 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c5235fa1-69c1-469b-88c1-d42d0b176666 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2741c615871c4ac59304338e4e746b60'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:03.809523 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c5235fa1-69c1-469b-88c1-d42d0b176666 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2741c615871c4ac59304338e4e746b60'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:03.824741 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c5235fa1-69c1-469b-88c1-d42d0b176666 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2741c615871c4ac59304338e4e746b60'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:03.825691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c5235fa1-69c1-469b-88c1-d42d0b176666 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2741c615871c4ac59304338e4e746b60'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:03.925854 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c5235fa1-69c1-469b-88c1-d42d0b176666 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2741c615871c4ac59304338e4e746b60'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:03.956176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1307/2608] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:41:03 2022] DELETE /identity/v3/users/2741c615871c4ac59304338e4e746b60 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:41:03.960658 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ecb5e-fb5a-4919-95ab-5c4266477834 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:03.961194 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ecb5e-fb5a-4919-95ab-5c4266477834 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:03.961194 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961ecb5e-fb5a-4919-95ab-5c4266477834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:03.980464 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-961ecb5e-fb5a-4919-95ab-5c4266477834 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:41:03.981996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-961ecb5e-fb5a-4919-95ab-5c4266477834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:04.023153 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:41:04.065405 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1302/2609] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:41:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:41:04.079238 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5235fa1-69c1-469b-88c1-d42d0b176666 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:04.081989 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c5235fa1-69c1-469b-88c1-d42d0b176666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:04.170449 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5235fa1-69c1-469b-88c1-d42d0b176666 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:04.175582 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581cc7f-fe7e-44d5-bf4b-76871ca2179a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:04.193898 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581cc7f-fe7e-44d5-bf4b-76871ca2179a service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:04.194209 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581cc7f-fe7e-44d5-bf4b-76871ca2179a service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:04.194523 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581cc7f-fe7e-44d5-bf4b-76871ca2179a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:04.210943 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581cc7f-fe7e-44d5-bf4b-76871ca2179a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:04.213157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581cc7f-fe7e-44d5-bf4b-76871ca2179a service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:04.248200 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1308/2610] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:41:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:41:04.977657 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ecb5e-fb5a-4919-95ab-5c4266477834 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:04.979358 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-961ecb5e-fb5a-4919-95ab-5c4266477834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:04.992053 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961ecb5e-fb5a-4919-95ab-5c4266477834 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:04.999305 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15073ce3-84bd-4e59-80d3-e5f0b0c453de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:05.000505 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15073ce3-84bd-4e59-80d3-e5f0b0c453de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:05.000963 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15073ce3-84bd-4e59-80d3-e5f0b0c453de admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:05.001406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15073ce3-84bd-4e59-80d3-e5f0b0c453de admin admin] PATH_INFO: `/v3/projects/37e1fb79754c485f94d8a0c814760cf1` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:05.003064 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15073ce3-84bd-4e59-80d3-e5f0b0c453de admin admin] RBAC: Authorizing `identity:delete_project(project_id=37e1fb79754c485f94d8a0c814760cf1)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:05.005216 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15073ce3-84bd-4e59-80d3-e5f0b0c453de admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:05.163174 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-15073ce3-84bd-4e59-80d3-e5f0b0c453de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37e1fb79754c485f94d8a0c814760cf1', 'request_id': 'req-15073ce3-84bd-4e59-80d3-e5f0b0c453de'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:05.171164 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-15073ce3-84bd-4e59-80d3-e5f0b0c453de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37e1fb79754c485f94d8a0c814760cf1', 'request_id': 'req-15073ce3-84bd-4e59-80d3-e5f0b0c453de'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:05.179163 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-15073ce3-84bd-4e59-80d3-e5f0b0c453de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37e1fb79754c485f94d8a0c814760cf1', 'request_id': 'req-15073ce3-84bd-4e59-80d3-e5f0b0c453de'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:05.294130 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-15073ce3-84bd-4e59-80d3-e5f0b0c453de admin admin] The token cache is being invalidate because project 37e1fb79754c485f94d8a0c814760cf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:41:05.294364 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-15073ce3-84bd-4e59-80d3-e5f0b0c453de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:05.294944 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-15073ce3-84bd-4e59-80d3-e5f0b0c453de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:05.297410 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1303/2611] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:41:04 2022] DELETE /identity/v3/projects/37e1fb79754c485f94d8a0c814760cf1 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:41:05.301537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581cc7f-fe7e-44d5-bf4b-76871ca2179a service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:05.312376 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581cc7f-fe7e-44d5-bf4b-76871ca2179a service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:05.329430 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:41:05.383820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8103777d-95d8-4fa5-a0f0-73bff422dd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:05.385122 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103777d-95d8-4fa5-a0f0-73bff422dd5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:05.385672 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103777d-95d8-4fa5-a0f0-73bff422dd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:05.386204 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8103777d-95d8-4fa5-a0f0-73bff422dd5b admin admin] PATH_INFO: `/v3/users/b647b2790d27430fae9f55e04ec7a847` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:05.387636 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103777d-95d8-4fa5-a0f0-73bff422dd5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b647b2790d27430fae9f55e04ec7a847)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:05.389830 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8103777d-95d8-4fa5-a0f0-73bff422dd5b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:05.611398 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8103777d-95d8-4fa5-a0f0-73bff422dd5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b647b2790d27430fae9f55e04ec7a847'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:05.612040 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8103777d-95d8-4fa5-a0f0-73bff422dd5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b647b2790d27430fae9f55e04ec7a847'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:05.620958 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8103777d-95d8-4fa5-a0f0-73bff422dd5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b647b2790d27430fae9f55e04ec7a847'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:05.622270 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8103777d-95d8-4fa5-a0f0-73bff422dd5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b647b2790d27430fae9f55e04ec7a847'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:05.691633 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-8103777d-95d8-4fa5-a0f0-73bff422dd5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b647b2790d27430fae9f55e04ec7a847'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:05.718510 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1309/2612] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:41:05 2022] DELETE /identity/v3/users/b647b2790d27430fae9f55e04ec7a847 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:41:06.492310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15073ce3-84bd-4e59-80d3-e5f0b0c453de admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:06.493766 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-15073ce3-84bd-4e59-80d3-e5f0b0c453de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:06.515380 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15073ce3-84bd-4e59-80d3-e5f0b0c453de admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:06.608264 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d69bdb5-0d62-45ca-a0e4-3a0f28a89ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:06.609457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69bdb5-0d62-45ca-a0e4-3a0f28a89ece admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:06.610044 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69bdb5-0d62-45ca-a0e4-3a0f28a89ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:06.610651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d69bdb5-0d62-45ca-a0e4-3a0f28a89ece admin admin] PATH_INFO: `/v3/projects/eb03759583a546f48e954ac025ef935d` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:06.612019 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d69bdb5-0d62-45ca-a0e4-3a0f28a89ece admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb03759583a546f48e954ac025ef935d)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:06.614546 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d69bdb5-0d62-45ca-a0e4-3a0f28a89ece admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:06.754817 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2d69bdb5-0d62-45ca-a0e4-3a0f28a89ece admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb03759583a546f48e954ac025ef935d', 'request_id': 'req-2d69bdb5-0d62-45ca-a0e4-3a0f28a89ece'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:06.767696 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2d69bdb5-0d62-45ca-a0e4-3a0f28a89ece admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb03759583a546f48e954ac025ef935d', 'request_id': 'req-2d69bdb5-0d62-45ca-a0e4-3a0f28a89ece'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:06.770375 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:41:06.777346 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2d69bdb5-0d62-45ca-a0e4-3a0f28a89ece admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb03759583a546f48e954ac025ef935d', 'request_id': 'req-2d69bdb5-0d62-45ca-a0e4-3a0f28a89ece'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:06.888299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2d69bdb5-0d62-45ca-a0e4-3a0f28a89ece admin admin] The token cache is being invalidate because project eb03759583a546f48e954ac025ef935d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:41:06.888299 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2d69bdb5-0d62-45ca-a0e4-3a0f28a89ece admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:06.889664 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2d69bdb5-0d62-45ca-a0e4-3a0f28a89ece admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:06.896476 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1304/2613] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:41:06 2022] DELETE /identity/v3/projects/eb03759583a546f48e954ac025ef935d => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:41:06.898558 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b4245-4c9c-4c60-a38c-b6c111711e28 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:06.899128 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b4245-4c9c-4c60-a38c-b6c111711e28 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:06.899619 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2b4245-4c9c-4c60-a38c-b6c111711e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:06.963152 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7c2b4245-4c9c-4c60-a38c-b6c111711e28 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:41:06.965147 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7c2b4245-4c9c-4c60-a38c-b6c111711e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:07.082462 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1310/2614] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:41:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:41:07.084792 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba610d2b-5b37-415a-9413-5c9f61610dad None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:07.085318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba610d2b-5b37-415a-9413-5c9f61610dad None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:07.085899 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba610d2b-5b37-415a-9413-5c9f61610dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:07.108984 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-ba610d2b-5b37-415a-9413-5c9f61610dad None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:41:07.110844 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-ba610d2b-5b37-415a-9413-5c9f61610dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:07.124895 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1305/2615] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:41:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:41:07.126318 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2b4245-4c9c-4c60-a38c-b6c111711e28 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:07.140697 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2b4245-4c9c-4c60-a38c-b6c111711e28 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:07.147385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed809190-410d-4f93-92b6-6059dd1617c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:07.148294 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed809190-410d-4f93-92b6-6059dd1617c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:07.148587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed809190-410d-4f93-92b6-6059dd1617c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:07.149784 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed809190-410d-4f93-92b6-6059dd1617c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:07.150060 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed809190-410d-4f93-92b6-6059dd1617c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:41:07.150551 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed809190-410d-4f93-92b6-6059dd1617c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:07.154709 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed809190-410d-4f93-92b6-6059dd1617c3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:07.177038 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1311/2616] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:41:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:41:07.178745 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba610d2b-5b37-415a-9413-5c9f61610dad None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:07.203185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba610d2b-5b37-415a-9413-5c9f61610dad None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:07.218262 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d31db2e-b0d3-469d-b464-a803d4dfef40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:07.219014 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31db2e-b0d3-469d-b464-a803d4dfef40 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:07.219472 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31db2e-b0d3-469d-b464-a803d4dfef40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:07.219761 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31db2e-b0d3-469d-b464-a803d4dfef40 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:07.225046 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d31db2e-b0d3-469d-b464-a803d4dfef40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:07.228612 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d31db2e-b0d3-469d-b464-a803d4dfef40 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:07.357326 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1306/2617] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:41:07 2022] POST /identity/v3/projects => generated 407 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:41:07.358518 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed809190-410d-4f93-92b6-6059dd1617c3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:07.369833 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed809190-410d-4f93-92b6-6059dd1617c3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:07.398810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:41:07.453986 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c750d7d-133a-4779-8732-9ed0d779b803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:07.455914 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c750d7d-133a-4779-8732-9ed0d779b803 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:07.456389 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c750d7d-133a-4779-8732-9ed0d779b803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:07.456613 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c750d7d-133a-4779-8732-9ed0d779b803 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:07.458117 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c750d7d-133a-4779-8732-9ed0d779b803 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:07.461614 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c750d7d-133a-4779-8732-9ed0d779b803 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:07.574238 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1312/2618] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:41:07 2022] POST /identity/v3/users => generated 304 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:41:07.578767 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d31db2e-b0d3-469d-b464-a803d4dfef40 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:07.597446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d31db2e-b0d3-469d-b464-a803d4dfef40 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:07.607147 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ea9be-8e60-4dec-aa06-1bc6320d2c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:07.608395 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ea9be-8e60-4dec-aa06-1bc6320d2c62 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:07.608908 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ea9be-8e60-4dec-aa06-1bc6320d2c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:07.609472 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82ea9be-8e60-4dec-aa06-1bc6320d2c62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:07.610741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82ea9be-8e60-4dec-aa06-1bc6320d2c62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:07.613849 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82ea9be-8e60-4dec-aa06-1bc6320d2c62 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:07.631202 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1307/2619] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:41:07.634372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c750d7d-133a-4779-8732-9ed0d779b803 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:07.650296 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c750d7d-133a-4779-8732-9ed0d779b803 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:07.658250 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819900a-a1d6-489b-81a0-b2f47560daf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:07.659815 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819900a-a1d6-489b-81a0-b2f47560daf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:07.661226 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819900a-a1d6-489b-81a0-b2f47560daf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:07.661741 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819900a-a1d6-489b-81a0-b2f47560daf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:07.664029 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b819900a-a1d6-489b-81a0-b2f47560daf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:07.666395 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b819900a-a1d6-489b-81a0-b2f47560daf6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:07.682060 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1313/2620] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:41:07 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:41:07.688105 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ea9be-8e60-4dec-aa06-1bc6320d2c62 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:07.706350 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82ea9be-8e60-4dec-aa06-1bc6320d2c62 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:07.718185 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50802a13-dbcc-4997-89c7-d5ea6c42b4a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:07.719330 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50802a13-dbcc-4997-89c7-d5ea6c42b4a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:07.719598 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50802a13-dbcc-4997-89c7-d5ea6c42b4a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:07.719875 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50802a13-dbcc-4997-89c7-d5ea6c42b4a2 admin admin] PATH_INFO: `/v3/projects/4fd04c7c5de34ceaab0daa8f4b747b60/users/5fd83455b7be4e1f82c76da26be810cc/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:07.757045 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50802a13-dbcc-4997-89c7-d5ea6c42b4a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fd04c7c5de34ceaab0daa8f4b747b60, user_id=5fd83455b7be4e1f82c76da26be810cc, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:07.759876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50802a13-dbcc-4997-89c7-d5ea6c42b4a2 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:07.885042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1308/2621] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:41:07 2022] PUT /identity/v3/projects/4fd04c7c5de34ceaab0daa8f4b747b60/users/5fd83455b7be4e1f82c76da26be810cc/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:41:07.895698 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be465b-22a9-4b2c-b7af-bba352c891b0 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:07.896001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be465b-22a9-4b2c-b7af-bba352c891b0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:07.896330 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20be465b-22a9-4b2c-b7af-bba352c891b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:07.939417 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-20be465b-22a9-4b2c-b7af-bba352c891b0 None None] MFA Rules not processed for user `5fd83455b7be4e1f82c76da26be810cc`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:41:07.941061 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-20be465b-22a9-4b2c-b7af-bba352c891b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:08.044076 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1314/2622] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:41:07 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:41:08.050257 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50802a13-dbcc-4997-89c7-d5ea6c42b4a2 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:08.115783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50802a13-dbcc-4997-89c7-d5ea6c42b4a2 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:08.122314 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c48a5-2820-4798-a02b-f2e7a7cca12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:08.123025 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c48a5-2820-4798-a02b-f2e7a7cca12e admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:08.123365 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c48a5-2820-4798-a02b-f2e7a7cca12e admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:08.123651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c48a5-2820-4798-a02b-f2e7a7cca12e admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:08.124346 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c48a5-2820-4798-a02b-f2e7a7cca12e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:08.126537 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c48a5-2820-4798-a02b-f2e7a7cca12e admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:08.520317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1309/2623] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:41:08 2022] POST /identity/v3/projects => generated 407 bytes in 471 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:41:08.523502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be465b-22a9-4b2c-b7af-bba352c891b0 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:08.536331 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20be465b-22a9-4b2c-b7af-bba352c891b0 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:08.597365 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b9ec0-922b-417d-a27d-edd5d014f122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:08.598140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b9ec0-922b-417d-a27d-edd5d014f122 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:08.598479 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b9ec0-922b-417d-a27d-edd5d014f122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:08.598755 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b9ec0-922b-417d-a27d-edd5d014f122 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:08.599619 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b9ec0-922b-417d-a27d-edd5d014f122 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:08.601565 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b9ec0-922b-417d-a27d-edd5d014f122 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:08.689942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1315/2624] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:41:08 2022] POST /identity/v3/users => generated 310 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:41:08.690919 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c48a5-2820-4798-a02b-f2e7a7cca12e admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:08.702271 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c48a5-2820-4798-a02b-f2e7a7cca12e admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:08.708120 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5321c5-5705-4260-b36b-2426bc5d8c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:08.708999 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5321c5-5705-4260-b36b-2426bc5d8c2c admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:08.709457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5321c5-5705-4260-b36b-2426bc5d8c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:08.709887 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5321c5-5705-4260-b36b-2426bc5d8c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:08.711474 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5321c5-5705-4260-b36b-2426bc5d8c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:08.713256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5321c5-5705-4260-b36b-2426bc5d8c2c admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:08.726634 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1310/2625] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:41:08.730739 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b9ec0-922b-417d-a27d-edd5d014f122 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:08.747123 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b9ec0-922b-417d-a27d-edd5d014f122 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:08.753050 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6374eb-0306-4709-81d4-2e206190d9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:08.754122 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6374eb-0306-4709-81d4-2e206190d9bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:08.754567 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6374eb-0306-4709-81d4-2e206190d9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:08.755035 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6374eb-0306-4709-81d4-2e206190d9bd admin admin] PATH_INFO: `/v3/projects/c1c0a1a9fa4b4ed1b6c8a124bce44f56/users/f1fea9add9554116953c5a55c2a31754/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:08.777556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6374eb-0306-4709-81d4-2e206190d9bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1c0a1a9fa4b4ed1b6c8a124bce44f56, user_id=f1fea9add9554116953c5a55c2a31754, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:08.780445 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6374eb-0306-4709-81d4-2e206190d9bd admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:08.863798 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:41:08.870652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1316/2626] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:41:08 2022] PUT /identity/v3/projects/c1c0a1a9fa4b4ed1b6c8a124bce44f56/users/f1fea9add9554116953c5a55c2a31754/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:41:08.874852 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5321c5-5705-4260-b36b-2426bc5d8c2c admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:08.890901 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5321c5-5705-4260-b36b-2426bc5d8c2c admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:09.027333 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:41:09.034856 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d8e6b-7880-479b-92ed-55e167206fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:09.035716 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d8e6b-7880-479b-92ed-55e167206fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:09.036479 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d8e6b-7880-479b-92ed-55e167206fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:09.036831 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d8e6b-7880-479b-92ed-55e167206fc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:09.038152 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d8e6b-7880-479b-92ed-55e167206fc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:09.039975 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d8e6b-7880-479b-92ed-55e167206fc4 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:09.052568 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1311/2627] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:41:08 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:41:09.056697 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6374eb-0306-4709-81d4-2e206190d9bd admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:09.070259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6374eb-0306-4709-81d4-2e206190d9bd admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:09.078482 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263cbf8-43a6-4a4b-847a-b1faf1c80131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:09.078482 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263cbf8-43a6-4a4b-847a-b1faf1c80131 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:09.078482 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263cbf8-43a6-4a4b-847a-b1faf1c80131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:09.078482 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263cbf8-43a6-4a4b-847a-b1faf1c80131 admin admin] PATH_INFO: `/v3/projects/c1c0a1a9fa4b4ed1b6c8a124bce44f56/users/f1fea9add9554116953c5a55c2a31754/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:09.084186 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263cbf8-43a6-4a4b-847a-b1faf1c80131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1c0a1a9fa4b4ed1b6c8a124bce44f56, user_id=f1fea9add9554116953c5a55c2a31754, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:09.087148 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263cbf8-43a6-4a4b-847a-b1faf1c80131 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:09.107662 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1317/2628] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:41:09 2022] PUT /identity/v3/projects/c1c0a1a9fa4b4ed1b6c8a124bce44f56/users/f1fea9add9554116953c5a55c2a31754/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:41:09.120032 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9104cbb-c331-43b2-8db0-3de51850e9e8 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:09.120404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9104cbb-c331-43b2-8db0-3de51850e9e8 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:09.120596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9104cbb-c331-43b2-8db0-3de51850e9e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:09.176218 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-c9104cbb-c331-43b2-8db0-3de51850e9e8 None None] MFA Rules not processed for user `f1fea9add9554116953c5a55c2a31754`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:41:09.176218 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-c9104cbb-c331-43b2-8db0-3de51850e9e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:09.279682 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1312/2629] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:41:09 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:41:09.288569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263cbf8-43a6-4a4b-847a-b1faf1c80131 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:09.290999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9263cbf8-43a6-4a4b-847a-b1faf1c80131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:09.360275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263cbf8-43a6-4a4b-847a-b1faf1c80131 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:09.365128 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34702680-f147-4ec8-b540-8c6a18c76481 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:09.365929 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34702680-f147-4ec8-b540-8c6a18c76481 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:09.366516 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34702680-f147-4ec8-b540-8c6a18c76481 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:09.366903 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34702680-f147-4ec8-b540-8c6a18c76481 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:09.386048 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34702680-f147-4ec8-b540-8c6a18c76481 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:09.387397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34702680-f147-4ec8-b540-8c6a18c76481 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:09.403927 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1318/2630] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:41:09 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:41:09.853282 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9104cbb-c331-43b2-8db0-3de51850e9e8 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:09.864941 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9104cbb-c331-43b2-8db0-3de51850e9e8 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:09.869455 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796202bd-e510-498d-8dec-16c547eb6698 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:09.870374 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796202bd-e510-498d-8dec-16c547eb6698 service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:09.870490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796202bd-e510-498d-8dec-16c547eb6698 service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:09.870860 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796202bd-e510-498d-8dec-16c547eb6698 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:09.945255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796202bd-e510-498d-8dec-16c547eb6698 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:09.948450 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796202bd-e510-498d-8dec-16c547eb6698 service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:09.971699 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1313/2631] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:41:09 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:41:19.395656 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=32,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:41:19.953147 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=66 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:41:20.431344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34702680-f147-4ec8-b540-8c6a18c76481 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:20.495855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34702680-f147-4ec8-b540-8c6a18c76481 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:20.501846 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8400d5-d197-40f6-ae78-b905a2b4b91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:20.502503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8400d5-d197-40f6-ae78-b905a2b4b91e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:20.502893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8400d5-d197-40f6-ae78-b905a2b4b91e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:20.503413 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8400d5-d197-40f6-ae78-b905a2b4b91e admin admin] PATH_INFO: `/v3/users/5fd83455b7be4e1f82c76da26be810cc` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:20.504766 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8400d5-d197-40f6-ae78-b905a2b4b91e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fd83455b7be4e1f82c76da26be810cc)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:20.506881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8400d5-d197-40f6-ae78-b905a2b4b91e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:20.698186 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7f8400d5-d197-40f6-ae78-b905a2b4b91e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fd83455b7be4e1f82c76da26be810cc'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:20.698830 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7f8400d5-d197-40f6-ae78-b905a2b4b91e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fd83455b7be4e1f82c76da26be810cc'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:20.706401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7f8400d5-d197-40f6-ae78-b905a2b4b91e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fd83455b7be4e1f82c76da26be810cc'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:20.707018 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7f8400d5-d197-40f6-ae78-b905a2b4b91e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fd83455b7be4e1f82c76da26be810cc'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:20.790982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7f8400d5-d197-40f6-ae78-b905a2b4b91e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fd83455b7be4e1f82c76da26be810cc'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:20.810428 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1319/2632] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:41:20 2022] DELETE /identity/v3/users/5fd83455b7be4e1f82c76da26be810cc => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:41:20.816229 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe31603-f4a9-4055-95f1-6ca5d4c338e5 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:20.816525 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe31603-f4a9-4055-95f1-6ca5d4c338e5 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:20.816812 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe31603-f4a9-4055-95f1-6ca5d4c338e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:20.822211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307e727-f53f-4831-b35c-aaba735de1f0 None None] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:20.822506 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307e727-f53f-4831-b35c-aaba735de1f0 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:20.822786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307e727-f53f-4831-b35c-aaba735de1f0 None None] PATH_INFO: `/` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:20.824438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1320/2633] 10.222.0.47 () {54 vars in 921 bytes} [Thu Jun 30 10:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 30 10:41:20.832526 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e920e1-ce6c-4d41-b554-d651e42f8fc9 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:20.832890 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e920e1-ce6c-4d41-b554-d651e42f8fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:20.833239 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e920e1-ce6c-4d41-b554-d651e42f8fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:20.835913 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-afe31603-f4a9-4055-95f1-6ca5d4c338e5 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:41:20.837342 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-afe31603-f4a9-4055-95f1-6ca5d4c338e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:20.853938 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-53e920e1-ce6c-4d41-b554-d651e42f8fc9 None None] MFA Rules not processed for user `71f6d7715e6f49ffb9fdf6e1307bbd51`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:41:20.855674 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-53e920e1-ce6c-4d41-b554-d651e42f8fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:20.903889 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1314/2634] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:41:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:41:20.914520 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe31603-f4a9-4055-95f1-6ca5d4c338e5 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:20.916248 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-afe31603-f4a9-4055-95f1-6ca5d4c338e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:20.929555 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1321/2635] 10.222.0.47 () {58 vars in 1018 bytes} [Thu Jun 30 10:41:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:41:20.941945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e920e1-ce6c-4d41-b554-d651e42f8fc9 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:20.943342 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-53e920e1-ce6c-4d41-b554-d651e42f8fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:20.960017 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:41:20.984388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe31603-f4a9-4055-95f1-6ca5d4c338e5 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:20.988402 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd16fed-1ef4-45c7-989f-8d1a90ec54cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:20.989136 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd16fed-1ef4-45c7-989f-8d1a90ec54cc service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:20.989513 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd16fed-1ef4-45c7-989f-8d1a90ec54cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:20.989888 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd16fed-1ef4-45c7-989f-8d1a90ec54cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:20.999485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd16fed-1ef4-45c7-989f-8d1a90ec54cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:21.001843 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd16fed-1ef4-45c7-989f-8d1a90ec54cc service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:21.013519 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1315/2636] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:41:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:41:21.030949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e920e1-ce6c-4d41-b554-d651e42f8fc9 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:21.035558 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833759ac-f23e-41f0-91c9-45cfb98dcec3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7a61d85908d42698b27b9fc00daa832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:21.036589 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833759ac-f23e-41f0-91c9-45cfb98dcec3 service glance] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:21.036941 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833759ac-f23e-41f0-91c9-45cfb98dcec3 service glance] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:21.037384 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833759ac-f23e-41f0-91c9-45cfb98dcec3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:21.049015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833759ac-f23e-41f0-91c9-45cfb98dcec3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:21.055095 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833759ac-f23e-41f0-91c9-45cfb98dcec3 service glance] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:21.062700 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1322/2637] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:41:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:41:21.554137 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd16fed-1ef4-45c7-989f-8d1a90ec54cc service neutron] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:21.555484 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-4fd16fed-1ef4-45c7-989f-8d1a90ec54cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:21.572740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd16fed-1ef4-45c7-989f-8d1a90ec54cc service neutron] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:21.584330 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7444a-405e-4f7c-a579-1a5d88728498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:21.584330 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7444a-405e-4f7c-a579-1a5d88728498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:21.584330 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7444a-405e-4f7c-a579-1a5d88728498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:21.584330 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef7444a-405e-4f7c-a579-1a5d88728498 admin admin] PATH_INFO: `/v3/projects/4fd04c7c5de34ceaab0daa8f4b747b60` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:21.585123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7444a-405e-4f7c-a579-1a5d88728498 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fd04c7c5de34ceaab0daa8f4b747b60)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:21.585123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef7444a-405e-4f7c-a579-1a5d88728498 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:21.719929 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-5ef7444a-405e-4f7c-a579-1a5d88728498 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fd04c7c5de34ceaab0daa8f4b747b60', 'request_id': 'req-5ef7444a-405e-4f7c-a579-1a5d88728498'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:21.726141 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-5ef7444a-405e-4f7c-a579-1a5d88728498 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fd04c7c5de34ceaab0daa8f4b747b60', 'request_id': 'req-5ef7444a-405e-4f7c-a579-1a5d88728498'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:21.731614 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-5ef7444a-405e-4f7c-a579-1a5d88728498 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fd04c7c5de34ceaab0daa8f4b747b60', 'request_id': 'req-5ef7444a-405e-4f7c-a579-1a5d88728498'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:21.859124 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-5ef7444a-405e-4f7c-a579-1a5d88728498 admin admin] The token cache is being invalidate because project 4fd04c7c5de34ceaab0daa8f4b747b60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:41:21.859361 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-5ef7444a-405e-4f7c-a579-1a5d88728498 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:21.860625 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-5ef7444a-405e-4f7c-a579-1a5d88728498 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:21.862345 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1316/2638] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:41:21 2022] DELETE /identity/v3/projects/4fd04c7c5de34ceaab0daa8f4b747b60 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:41:21.866445 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833759ac-f23e-41f0-91c9-45cfb98dcec3 service glance] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:21.877589 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833759ac-f23e-41f0-91c9-45cfb98dcec3 service glance] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:21.936343 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c0c075-94d0-4ba6-b0c5-87eb65aaa282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:21.937331 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0c075-94d0-4ba6-b0c5-87eb65aaa282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:21.937628 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0c075-94d0-4ba6-b0c5-87eb65aaa282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:21.937957 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c0c075-94d0-4ba6-b0c5-87eb65aaa282 admin admin] PATH_INFO: `/v3/users/f1fea9add9554116953c5a55c2a31754` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:21.939929 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c0c075-94d0-4ba6-b0c5-87eb65aaa282 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1fea9add9554116953c5a55c2a31754)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:21.941896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c0c075-94d0-4ba6-b0c5-87eb65aaa282 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:22.145101 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e7c0c075-94d0-4ba6-b0c5-87eb65aaa282 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1fea9add9554116953c5a55c2a31754'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:22.145686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e7c0c075-94d0-4ba6-b0c5-87eb65aaa282 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1fea9add9554116953c5a55c2a31754'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:22.152638 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e7c0c075-94d0-4ba6-b0c5-87eb65aaa282 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1fea9add9554116953c5a55c2a31754'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:22.153271 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e7c0c075-94d0-4ba6-b0c5-87eb65aaa282 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1fea9add9554116953c5a55c2a31754'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:22.222478 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e7c0c075-94d0-4ba6-b0c5-87eb65aaa282 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1fea9add9554116953c5a55c2a31754'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:22.244116 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1323/2639] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:41:21 2022] DELETE /identity/v3/users/f1fea9add9554116953c5a55c2a31754 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:41:22.760535 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7444a-405e-4f7c-a579-1a5d88728498 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:22.761961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-5ef7444a-405e-4f7c-a579-1a5d88728498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:22.777792 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef7444a-405e-4f7c-a579-1a5d88728498 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:22.805377 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:41:22.836937 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b582a718-dcbc-499b-b9ec-a770c2dcaf88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:22.837798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582a718-dcbc-499b-b9ec-a770c2dcaf88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:22.838086 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582a718-dcbc-499b-b9ec-a770c2dcaf88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:22.838362 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b582a718-dcbc-499b-b9ec-a770c2dcaf88 admin admin] PATH_INFO: `/v3/projects/c1c0a1a9fa4b4ed1b6c8a124bce44f56` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:22.839660 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b582a718-dcbc-499b-b9ec-a770c2dcaf88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1c0a1a9fa4b4ed1b6c8a124bce44f56)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:22.841610 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b582a718-dcbc-499b-b9ec-a770c2dcaf88 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:22.971502 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-b582a718-dcbc-499b-b9ec-a770c2dcaf88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1c0a1a9fa4b4ed1b6c8a124bce44f56', 'request_id': 'req-b582a718-dcbc-499b-b9ec-a770c2dcaf88'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:22.978291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-b582a718-dcbc-499b-b9ec-a770c2dcaf88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1c0a1a9fa4b4ed1b6c8a124bce44f56', 'request_id': 'req-b582a718-dcbc-499b-b9ec-a770c2dcaf88'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:22.983847 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-b582a718-dcbc-499b-b9ec-a770c2dcaf88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1c0a1a9fa4b4ed1b6c8a124bce44f56', 'request_id': 'req-b582a718-dcbc-499b-b9ec-a770c2dcaf88'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:23.065016 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-b582a718-dcbc-499b-b9ec-a770c2dcaf88 admin admin] The token cache is being invalidate because project c1c0a1a9fa4b4ed1b6c8a124bce44f56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:41:23.065220 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-b582a718-dcbc-499b-b9ec-a770c2dcaf88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:23.065889 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-b582a718-dcbc-499b-b9ec-a770c2dcaf88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:23.067541 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1317/2640] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:41:22 2022] DELETE /identity/v3/projects/c1c0a1a9fa4b4ed1b6c8a124bce44f56 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:41:23.072625 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d90f8b-4a76-4feb-a73b-ba7ed776bf04 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:23.073143 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d90f8b-4a76-4feb-a73b-ba7ed776bf04 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:23.073245 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d90f8b-4a76-4feb-a73b-ba7ed776bf04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:23.091586 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a9d90f8b-4a76-4feb-a73b-ba7ed776bf04 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:41:23.092828 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a9d90f8b-4a76-4feb-a73b-ba7ed776bf04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:23.102205 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:41:23.158274 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1324/2641] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:41:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:41:23.166746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614d6a3-b99f-429e-b46c-227862c8623a None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:23.167038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614d6a3-b99f-429e-b46c-227862c8623a None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:23.167347 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4614d6a3-b99f-429e-b46c-227862c8623a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:23.190183 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-4614d6a3-b99f-429e-b46c-227862c8623a None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:41:23.191492 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-4614d6a3-b99f-429e-b46c-227862c8623a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:23.199725 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1318/2642] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:41:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:41:23.203973 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d90f8b-4a76-4feb-a73b-ba7ed776bf04 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:23.215436 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d90f8b-4a76-4feb-a73b-ba7ed776bf04 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:23.222022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec148c-5bcf-4861-972a-1f1a52896bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:23.222700 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec148c-5bcf-4861-972a-1f1a52896bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:23.222990 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec148c-5bcf-4861-972a-1f1a52896bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:23.223342 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ec148c-5bcf-4861-972a-1f1a52896bc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:23.224269 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec148c-5bcf-4861-972a-1f1a52896bc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:41:23.224603 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec148c-5bcf-4861-972a-1f1a52896bc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:23.226276 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ec148c-5bcf-4861-972a-1f1a52896bc1 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:23.243087 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1325/2643] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:41:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:41:23.244424 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614d6a3-b99f-429e-b46c-227862c8623a None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:23.254515 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4614d6a3-b99f-429e-b46c-227862c8623a None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:23.266426 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e125b57f-21b1-4106-856d-3de14d4222e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:23.266426 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125b57f-21b1-4106-856d-3de14d4222e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:23.266426 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125b57f-21b1-4106-856d-3de14d4222e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:23.266426 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e125b57f-21b1-4106-856d-3de14d4222e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:23.267029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e125b57f-21b1-4106-856d-3de14d4222e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:23.267029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e125b57f-21b1-4106-856d-3de14d4222e5 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:23.343470 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1319/2644] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:41:23 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:41:23.348632 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec148c-5bcf-4861-972a-1f1a52896bc1 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:23.359104 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ec148c-5bcf-4861-972a-1f1a52896bc1 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:23.416401 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c411d213-bdf4-4fb1-bd06-210f50a7b33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:23.417063 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411d213-bdf4-4fb1-bd06-210f50a7b33e admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:23.417346 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411d213-bdf4-4fb1-bd06-210f50a7b33e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:23.417621 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c411d213-bdf4-4fb1-bd06-210f50a7b33e admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:23.418301 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c411d213-bdf4-4fb1-bd06-210f50a7b33e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:23.420254 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c411d213-bdf4-4fb1-bd06-210f50a7b33e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:23.531675 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1326/2645] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:41:23 2022] POST /identity/v3/users => generated 287 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:41:23.536791 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e125b57f-21b1-4106-856d-3de14d4222e5 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:23.547713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e125b57f-21b1-4106-856d-3de14d4222e5 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:23.552722 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5496ffc-087c-40e6-bb48-abba560814ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:23.552984 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5496ffc-087c-40e6-bb48-abba560814ce admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:23.553204 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5496ffc-087c-40e6-bb48-abba560814ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:23.553462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5496ffc-087c-40e6-bb48-abba560814ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:23.554186 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5496ffc-087c-40e6-bb48-abba560814ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:23.555805 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5496ffc-087c-40e6-bb48-abba560814ce admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:23.566253 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1320/2646] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:41:23.570301 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c411d213-bdf4-4fb1-bd06-210f50a7b33e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:23.581345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c411d213-bdf4-4fb1-bd06-210f50a7b33e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:23.586101 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebde7d9-b5d0-4a4a-bb90-cce15468003c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:23.586765 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebde7d9-b5d0-4a4a-bb90-cce15468003c admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:23.587127 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebde7d9-b5d0-4a4a-bb90-cce15468003c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:23.587439 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebde7d9-b5d0-4a4a-bb90-cce15468003c admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:23.588171 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebde7d9-b5d0-4a4a-bb90-cce15468003c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:23.589783 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebde7d9-b5d0-4a4a-bb90-cce15468003c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:23.608336 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1327/2647] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:41:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:41:23.608507 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5496ffc-087c-40e6-bb48-abba560814ce admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:23.615886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5496ffc-087c-40e6-bb48-abba560814ce admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:23.621165 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b5c826-0e9a-4680-8590-d15cb8dc47ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:23.622123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5c826-0e9a-4680-8590-d15cb8dc47ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:23.622475 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5c826-0e9a-4680-8590-d15cb8dc47ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:23.622822 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b5c826-0e9a-4680-8590-d15cb8dc47ea admin admin] PATH_INFO: `/v3/projects/a5187ef60c014ab5bfcad4ca1f90bc72/users/abe1d422c8b244d9821701925a357a77/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:23.638930 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b5c826-0e9a-4680-8590-d15cb8dc47ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5187ef60c014ab5bfcad4ca1f90bc72, user_id=abe1d422c8b244d9821701925a357a77, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:23.642842 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b5c826-0e9a-4680-8590-d15cb8dc47ea admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:23.720704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1321/2648] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:41:23 2022] PUT /identity/v3/projects/a5187ef60c014ab5bfcad4ca1f90bc72/users/abe1d422c8b244d9821701925a357a77/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:41:23.732090 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843146ce-2047-4f53-8050-750a512c54ab None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:23.734454 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843146ce-2047-4f53-8050-750a512c54ab None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:23.734454 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843146ce-2047-4f53-8050-750a512c54ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:23.760906 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-843146ce-2047-4f53-8050-750a512c54ab None None] MFA Rules not processed for user `abe1d422c8b244d9821701925a357a77`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:41:23.763140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-843146ce-2047-4f53-8050-750a512c54ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:23.834229 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1328/2649] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:41:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:41:23.845256 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b5c826-0e9a-4680-8590-d15cb8dc47ea admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:23.847014 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-12b5c826-0e9a-4680-8590-d15cb8dc47ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:23.904993 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b5c826-0e9a-4680-8590-d15cb8dc47ea admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:23.909156 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd438dd2-c815-44f5-a4b1-50f750b575f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:23.909800 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd438dd2-c815-44f5-a4b1-50f750b575f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:23.910081 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd438dd2-c815-44f5-a4b1-50f750b575f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:23.910355 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd438dd2-c815-44f5-a4b1-50f750b575f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:23.916768 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:41:23.919004 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd438dd2-c815-44f5-a4b1-50f750b575f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:23.920450 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd438dd2-c815-44f5-a4b1-50f750b575f4 service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:23.929547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1322/2650] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:41:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:41:23.951112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843146ce-2047-4f53-8050-750a512c54ab None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:23.961713 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:41:24.014532 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843146ce-2047-4f53-8050-750a512c54ab None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:24.020204 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c825051-6ed2-4e2e-b523-73067a8c9611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:24.021084 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c825051-6ed2-4e2e-b523-73067a8c9611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:24.021364 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c825051-6ed2-4e2e-b523-73067a8c9611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:24.021635 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c825051-6ed2-4e2e-b523-73067a8c9611 admin admin] PATH_INFO: `/v3/users/abe1d422c8b244d9821701925a357a77` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:24.022851 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c825051-6ed2-4e2e-b523-73067a8c9611 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abe1d422c8b244d9821701925a357a77)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:24.025074 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c825051-6ed2-4e2e-b523-73067a8c9611 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:24.256096 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7c825051-6ed2-4e2e-b523-73067a8c9611 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abe1d422c8b244d9821701925a357a77'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:24.257008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7c825051-6ed2-4e2e-b523-73067a8c9611 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abe1d422c8b244d9821701925a357a77'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:24.264576 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7c825051-6ed2-4e2e-b523-73067a8c9611 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abe1d422c8b244d9821701925a357a77'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:24.265502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7c825051-6ed2-4e2e-b523-73067a8c9611 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abe1d422c8b244d9821701925a357a77'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:24.356727 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-7c825051-6ed2-4e2e-b523-73067a8c9611 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abe1d422c8b244d9821701925a357a77'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:24.377575 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1329/2651] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:41:23 2022] DELETE /identity/v3/users/abe1d422c8b244d9821701925a357a77 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:41:24.382316 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416946f-2e2d-4b3b-84c3-5be18511bd4b None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:24.382616 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416946f-2e2d-4b3b-84c3-5be18511bd4b None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:24.382923 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416946f-2e2d-4b3b-84c3-5be18511bd4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:24.404574 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-5416946f-2e2d-4b3b-84c3-5be18511bd4b None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:41:24.405867 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-5416946f-2e2d-4b3b-84c3-5be18511bd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:24.470850 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1323/2652] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:41:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:41:24.480956 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c825051-6ed2-4e2e-b523-73067a8c9611 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:24.482438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7c825051-6ed2-4e2e-b523-73067a8c9611 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:24.546475 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c825051-6ed2-4e2e-b523-73067a8c9611 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:24.551053 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a4760-29ab-4072-b1a1-754e277ef7df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:24.551737 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a4760-29ab-4072-b1a1-754e277ef7df service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:24.552021 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a4760-29ab-4072-b1a1-754e277ef7df service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:24.552296 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a4760-29ab-4072-b1a1-754e277ef7df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:24.561587 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a4760-29ab-4072-b1a1-754e277ef7df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:24.563122 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a4760-29ab-4072-b1a1-754e277ef7df service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:24.572790 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1330/2653] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:41:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:41:25.136175 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416946f-2e2d-4b3b-84c3-5be18511bd4b None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:25.137740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-5416946f-2e2d-4b3b-84c3-5be18511bd4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:25.155979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416946f-2e2d-4b3b-84c3-5be18511bd4b None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:25.166745 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f5d3fc-94e9-4434-8b59-a6ac16e8bd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:25.167764 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5d3fc-94e9-4434-8b59-a6ac16e8bd68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:25.168048 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5d3fc-94e9-4434-8b59-a6ac16e8bd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:25.168323 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f5d3fc-94e9-4434-8b59-a6ac16e8bd68 admin admin] PATH_INFO: `/v3/projects/a5187ef60c014ab5bfcad4ca1f90bc72` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:25.169552 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5d3fc-94e9-4434-8b59-a6ac16e8bd68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5187ef60c014ab5bfcad4ca1f90bc72)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:25.172081 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f5d3fc-94e9-4434-8b59-a6ac16e8bd68 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:25.271013 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-c5f5d3fc-94e9-4434-8b59-a6ac16e8bd68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5187ef60c014ab5bfcad4ca1f90bc72', 'request_id': 'req-c5f5d3fc-94e9-4434-8b59-a6ac16e8bd68'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:25.277079 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-c5f5d3fc-94e9-4434-8b59-a6ac16e8bd68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5187ef60c014ab5bfcad4ca1f90bc72', 'request_id': 'req-c5f5d3fc-94e9-4434-8b59-a6ac16e8bd68'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:25.282912 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-c5f5d3fc-94e9-4434-8b59-a6ac16e8bd68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5187ef60c014ab5bfcad4ca1f90bc72', 'request_id': 'req-c5f5d3fc-94e9-4434-8b59-a6ac16e8bd68'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:25.373465 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-c5f5d3fc-94e9-4434-8b59-a6ac16e8bd68 admin admin] The token cache is being invalidate because project a5187ef60c014ab5bfcad4ca1f90bc72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:41:25.373846 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-c5f5d3fc-94e9-4434-8b59-a6ac16e8bd68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:25.374573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-c5f5d3fc-94e9-4434-8b59-a6ac16e8bd68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:25.376825 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1324/2654] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:41:25 2022] DELETE /identity/v3/projects/a5187ef60c014ab5bfcad4ca1f90bc72 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:41:25.381602 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8589d27-fd95-4423-92ff-12e64d773ea4 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:25.382105 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8589d27-fd95-4423-92ff-12e64d773ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:25.382188 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8589d27-fd95-4423-92ff-12e64d773ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:25.403345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b8589d27-fd95-4423-92ff-12e64d773ea4 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:41:25.406510 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b8589d27-fd95-4423-92ff-12e64d773ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:25.434443 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:41:25.469955 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1331/2655] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:41:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:41:25.477397 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ade9d8-3ae5-4e9f-b7b8-3f7efc146acb None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:25.477688 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ade9d8-3ae5-4e9f-b7b8-3f7efc146acb None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:25.477967 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ade9d8-3ae5-4e9f-b7b8-3f7efc146acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:25.497139 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-66ade9d8-3ae5-4e9f-b7b8-3f7efc146acb None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:41:25.503777 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-66ade9d8-3ae5-4e9f-b7b8-3f7efc146acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:25.507394 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1325/2656] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:41:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:41:25.511429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8589d27-fd95-4423-92ff-12e64d773ea4 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:25.522897 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8589d27-fd95-4423-92ff-12e64d773ea4 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:25.537242 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec90c8-d8e8-4a9d-b445-5b6a8ee73d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:25.539395 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec90c8-d8e8-4a9d-b445-5b6a8ee73d09 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:25.539395 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec90c8-d8e8-4a9d-b445-5b6a8ee73d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:25.539395 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caec90c8-d8e8-4a9d-b445-5b6a8ee73d09 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:25.540705 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caec90c8-d8e8-4a9d-b445-5b6a8ee73d09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:41:25.541169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caec90c8-d8e8-4a9d-b445-5b6a8ee73d09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:25.543508 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caec90c8-d8e8-4a9d-b445-5b6a8ee73d09 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:25.559610 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1332/2657] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:41:25.564692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ade9d8-3ae5-4e9f-b7b8-3f7efc146acb None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:25.574272 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ade9d8-3ae5-4e9f-b7b8-3f7efc146acb None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:25.580739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c7cd81-0c1c-4f7b-9fd5-3da882a9404e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:25.581421 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7cd81-0c1c-4f7b-9fd5-3da882a9404e admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:25.581712 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7cd81-0c1c-4f7b-9fd5-3da882a9404e admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:25.581993 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c7cd81-0c1c-4f7b-9fd5-3da882a9404e admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:25.582699 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c7cd81-0c1c-4f7b-9fd5-3da882a9404e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:25.584694 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c7cd81-0c1c-4f7b-9fd5-3da882a9404e admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:25.689672 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1326/2658] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:41:25 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:41:25.693234 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec90c8-d8e8-4a9d-b445-5b6a8ee73d09 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:25.707560 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caec90c8-d8e8-4a9d-b445-5b6a8ee73d09 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:25.763976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3f0e27-a153-4a1b-a76e-62a15fde5fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:25.764685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f0e27-a153-4a1b-a76e-62a15fde5fdc admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:25.764964 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f0e27-a153-4a1b-a76e-62a15fde5fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:25.765235 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3f0e27-a153-4a1b-a76e-62a15fde5fdc admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:25.765929 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3f0e27-a153-4a1b-a76e-62a15fde5fdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:25.767893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3f0e27-a153-4a1b-a76e-62a15fde5fdc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:25.917627 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1333/2659] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:41:25 2022] POST /identity/v3/users => generated 293 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:41:25.919391 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c7cd81-0c1c-4f7b-9fd5-3da882a9404e admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:25.930725 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c7cd81-0c1c-4f7b-9fd5-3da882a9404e admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:25.935875 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0713681-b272-40a5-b393-bcb11b27e0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:25.942905 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0713681-b272-40a5-b393-bcb11b27e0a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:25.943587 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0713681-b272-40a5-b393-bcb11b27e0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:25.943587 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0713681-b272-40a5-b393-bcb11b27e0a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:25.944549 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0713681-b272-40a5-b393-bcb11b27e0a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:25.946268 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0713681-b272-40a5-b393-bcb11b27e0a0 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:25.953886 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462acb6-eb21-4e71-9f97-7f0709616c60 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:25.954178 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462acb6-eb21-4e71-9f97-7f0709616c60 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:25.954480 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7462acb6-eb21-4e71-9f97-7f0709616c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:25.958358 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1327/2660] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:41:25.963577 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0713681-b272-40a5-b393-bcb11b27e0a0 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:25.974272 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0713681-b272-40a5-b393-bcb11b27e0a0 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:25.976723 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7462acb6-eb21-4e71-9f97-7f0709616c60 None None] MFA Rules not processed for user `e9cf01f5974c45c6bd19b3e51b23d89c`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:41:25.976723 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7462acb6-eb21-4e71-9f97-7f0709616c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:25.979437 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d4619-c6b5-425a-a7f3-13771cff7e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:25.980125 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d4619-c6b5-425a-a7f3-13771cff7e3d admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:25.980368 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d4619-c6b5-425a-a7f3-13771cff7e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:25.980662 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d4619-c6b5-425a-a7f3-13771cff7e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:25.981382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d4619-c6b5-425a-a7f3-13771cff7e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:25.983026 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d4619-c6b5-425a-a7f3-13771cff7e3d admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:25.995758 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1328/2661] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:41:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:41:25.997082 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076679d-498d-4263-a022-6e537c77f5a8 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:25.997444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076679d-498d-4263-a022-6e537c77f5a8 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:25.997794 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076679d-498d-4263-a022-6e537c77f5a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:26.016703 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-b076679d-498d-4263-a022-6e537c77f5a8 None None] MFA Rules not processed for user `66ad8defb9b6453c96f1a8a7343695ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:41:26.017945 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-b076679d-498d-4263-a022-6e537c77f5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:26.037807 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1334/2662] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:41:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:41:26.040236 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462acb6-eb21-4e71-9f97-7f0709616c60 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:26.042993 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:41:26.054056 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7462acb6-eb21-4e71-9f97-7f0709616c60 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:26.059665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d2616-9b1f-4f82-8fd7-4bebc56ff268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:26.063211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d2616-9b1f-4f82-8fd7-4bebc56ff268 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:26.063211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d2616-9b1f-4f82-8fd7-4bebc56ff268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:26.063211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d2616-9b1f-4f82-8fd7-4bebc56ff268 admin admin] PATH_INFO: `/v3/projects/b0ae1b556b4044938658e54cdbe7d5de/users/def773a6d1ea4406ae6a2054b6b4f258/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:26.077272 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1329/2663] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jun 30 10:41:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:41:26.078385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d2616-9b1f-4f82-8fd7-4bebc56ff268 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0ae1b556b4044938658e54cdbe7d5de, user_id=def773a6d1ea4406ae6a2054b6b4f258, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:26.081503 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d2616-9b1f-4f82-8fd7-4bebc56ff268 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:26.091005 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076679d-498d-4263-a022-6e537c77f5a8 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:26.092901 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-b076679d-498d-4263-a022-6e537c77f5a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:26.164606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076679d-498d-4263-a022-6e537c77f5a8 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:26.169883 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b429844f-c49f-4a0b-82c3-e9a6ed0bd852 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07de926cb8594b098f79111dab8a9cc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:26.170540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429844f-c49f-4a0b-82c3-e9a6ed0bd852 service swift] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:26.170832 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429844f-c49f-4a0b-82c3-e9a6ed0bd852 service swift] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:26.171114 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b429844f-c49f-4a0b-82c3-e9a6ed0bd852 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:26.180516 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b429844f-c49f-4a0b-82c3-e9a6ed0bd852 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:26.182114 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b429844f-c49f-4a0b-82c3-e9a6ed0bd852 service swift] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:26.191510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1330/2664] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jun 30 10:41:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:41:26.220099 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1335/2665] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:41:26 2022] PUT /identity/v3/projects/b0ae1b556b4044938658e54cdbe7d5de/users/def773a6d1ea4406ae6a2054b6b4f258/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:41:26.221999 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3855c0-b297-43e6-9d42-7cda0f62392b None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:26.222289 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3855c0-b297-43e6-9d42-7cda0f62392b None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:26.222567 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3855c0-b297-43e6-9d42-7cda0f62392b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:26.249150 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-db3855c0-b297-43e6-9d42-7cda0f62392b None None] MFA Rules not processed for user `def773a6d1ea4406ae6a2054b6b4f258`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:41:26.250908 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-db3855c0-b297-43e6-9d42-7cda0f62392b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:26.317285 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1331/2666] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:41:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:41:26.327776 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d2616-9b1f-4f82-8fd7-4bebc56ff268 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:26.330969 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-893d2616-9b1f-4f82-8fd7-4bebc56ff268 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:26.384176 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:41:26.449634 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d2616-9b1f-4f82-8fd7-4bebc56ff268 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:26.457545 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc7480-627c-48cf-8005-e3dfd67e3efc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:26.460223 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc7480-627c-48cf-8005-e3dfd67e3efc service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:26.460223 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc7480-627c-48cf-8005-e3dfd67e3efc service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:26.460223 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc7480-627c-48cf-8005-e3dfd67e3efc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:26.476320 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc7480-627c-48cf-8005-e3dfd67e3efc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:26.478693 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc7480-627c-48cf-8005-e3dfd67e3efc service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:26.512665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1336/2667] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:41:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:41:36.319484 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=76 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:41:36.493383 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=13 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:41:37.064637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3855c0-b297-43e6-9d42-7cda0f62392b None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:37.129802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3855c0-b297-43e6-9d42-7cda0f62392b None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:37.134129 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae226bf6-d751-41ea-9757-348522c0fd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:37.135001 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae226bf6-d751-41ea-9757-348522c0fd2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:37.135307 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae226bf6-d751-41ea-9757-348522c0fd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:37.135590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae226bf6-d751-41ea-9757-348522c0fd2a admin admin] PATH_INFO: `/v3/users/def773a6d1ea4406ae6a2054b6b4f258` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:37.136767 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae226bf6-d751-41ea-9757-348522c0fd2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=def773a6d1ea4406ae6a2054b6b4f258)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:37.138664 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae226bf6-d751-41ea-9757-348522c0fd2a admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:37.437700 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-ae226bf6-d751-41ea-9757-348522c0fd2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'def773a6d1ea4406ae6a2054b6b4f258'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:37.542228 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-ae226bf6-d751-41ea-9757-348522c0fd2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'def773a6d1ea4406ae6a2054b6b4f258'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:37.556479 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-ae226bf6-d751-41ea-9757-348522c0fd2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'def773a6d1ea4406ae6a2054b6b4f258'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:37.557210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-ae226bf6-d751-41ea-9757-348522c0fd2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'def773a6d1ea4406ae6a2054b6b4f258'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:37.557838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-ae226bf6-d751-41ea-9757-348522c0fd2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'def773a6d1ea4406ae6a2054b6b4f258'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:37.570732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1332/2668] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:41:37 2022] DELETE /identity/v3/users/def773a6d1ea4406ae6a2054b6b4f258 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:41:37.575795 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b71577-e52d-4587-a396-d7310dc12b6f None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:37.575795 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b71577-e52d-4587-a396-d7310dc12b6f None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:37.575795 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b71577-e52d-4587-a396-d7310dc12b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:37.595149 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f5b71577-e52d-4587-a396-d7310dc12b6f None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:41:37.596404 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f5b71577-e52d-4587-a396-d7310dc12b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:37.657214 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1337/2669] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:41:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:41:37.667786 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae226bf6-d751-41ea-9757-348522c0fd2a admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:37.669189 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-ae226bf6-d751-41ea-9757-348522c0fd2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:37.732328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae226bf6-d751-41ea-9757-348522c0fd2a admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:37.737865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8e42e8-f9ab-4154-8dd3-dc04ee7de10d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:37.738910 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e42e8-f9ab-4154-8dd3-dc04ee7de10d service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:37.738910 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e42e8-f9ab-4154-8dd3-dc04ee7de10d service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:37.739119 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8e42e8-f9ab-4154-8dd3-dc04ee7de10d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:37.749589 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e42e8-f9ab-4154-8dd3-dc04ee7de10d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:37.752692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8e42e8-f9ab-4154-8dd3-dc04ee7de10d service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:37.769597 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1333/2670] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:41:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:41:38.355833 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b71577-e52d-4587-a396-d7310dc12b6f None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:38.357465 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f5b71577-e52d-4587-a396-d7310dc12b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:38.372494 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b71577-e52d-4587-a396-d7310dc12b6f None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:38.379663 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987dbc17-ee38-4525-9a6a-f7bb95aaed6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:38.380649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987dbc17-ee38-4525-9a6a-f7bb95aaed6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:38.381495 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987dbc17-ee38-4525-9a6a-f7bb95aaed6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:38.381946 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987dbc17-ee38-4525-9a6a-f7bb95aaed6e admin admin] PATH_INFO: `/v3/projects/b0ae1b556b4044938658e54cdbe7d5de` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:38.383337 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987dbc17-ee38-4525-9a6a-f7bb95aaed6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0ae1b556b4044938658e54cdbe7d5de)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:38.385563 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987dbc17-ee38-4525-9a6a-f7bb95aaed6e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:38.479488 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-987dbc17-ee38-4525-9a6a-f7bb95aaed6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0ae1b556b4044938658e54cdbe7d5de', 'request_id': 'req-987dbc17-ee38-4525-9a6a-f7bb95aaed6e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:38.587721 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-987dbc17-ee38-4525-9a6a-f7bb95aaed6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0ae1b556b4044938658e54cdbe7d5de', 'request_id': 'req-987dbc17-ee38-4525-9a6a-f7bb95aaed6e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:38.595950 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-987dbc17-ee38-4525-9a6a-f7bb95aaed6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0ae1b556b4044938658e54cdbe7d5de', 'request_id': 'req-987dbc17-ee38-4525-9a6a-f7bb95aaed6e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:38.607878 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-987dbc17-ee38-4525-9a6a-f7bb95aaed6e admin admin] The token cache is being invalidate because project b0ae1b556b4044938658e54cdbe7d5de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:41:38.608188 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-987dbc17-ee38-4525-9a6a-f7bb95aaed6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:38.608952 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-987dbc17-ee38-4525-9a6a-f7bb95aaed6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:41:38.611326 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1338/2671] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:41:38 2022] DELETE /identity/v3/projects/b0ae1b556b4044938658e54cdbe7d5de => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:41:47.760265 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:41:47.877942 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf77191-7c89-4ad6-8ee5-1cf1e1b7e31f None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:47.878481 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf77191-7c89-4ad6-8ee5-1cf1e1b7e31f None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:47.878943 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf77191-7c89-4ad6-8ee5-1cf1e1b7e31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:47.897396 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-cbf77191-7c89-4ad6-8ee5-1cf1e1b7e31f None None] MFA Rules not processed for user `66ad8defb9b6453c96f1a8a7343695ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:41:47.898604 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-cbf77191-7c89-4ad6-8ee5-1cf1e1b7e31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:47.958342 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1334/2672] 10.222.0.47 () {58 vars in 967 bytes} [Thu Jun 30 10:41:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:41:47.971143 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987dbc17-ee38-4525-9a6a-f7bb95aaed6e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:47.981805 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-987dbc17-ee38-4525-9a6a-f7bb95aaed6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:48.042468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987dbc17-ee38-4525-9a6a-f7bb95aaed6e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:48.047598 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eb463b-df99-4a6d-a878-04fac753c3bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07de926cb8594b098f79111dab8a9cc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:48.048250 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb463b-df99-4a6d-a878-04fac753c3bd service swift] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:48.048553 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb463b-df99-4a6d-a878-04fac753c3bd service swift] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:48.048826 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eb463b-df99-4a6d-a878-04fac753c3bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:48.056856 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb463b-df99-4a6d-a878-04fac753c3bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:48.058471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eb463b-df99-4a6d-a878-04fac753c3bd service swift] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:48.067760 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1339/2673] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jun 30 10:41:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 30 10:41:55.306704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf77191-7c89-4ad6-8ee5-1cf1e1b7e31f None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:41:55.308146 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-cbf77191-7c89-4ad6-8ee5-1cf1e1b7e31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:55.384653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf77191-7c89-4ad6-8ee5-1cf1e1b7e31f None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:41:55.389594 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda7d8c-49a6-4614-81e8-14334755e2be service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46c99f43a9344d7d92632990e45bb9cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:41:55.390838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda7d8c-49a6-4614-81e8-14334755e2be service placement] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:41:55.391124 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda7d8c-49a6-4614-81e8-14334755e2be service placement] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:41:55.391439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda7d8c-49a6-4614-81e8-14334755e2be service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:41:55.399842 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-6bda7d8c-49a6-4614-81e8-14334755e2be service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:41:55.404925 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda7d8c-49a6-4614-81e8-14334755e2be service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:41:55.406534 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bda7d8c-49a6-4614-81e8-14334755e2be service placement] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:41:55.416887 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1335/2674] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:41:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:41:58.064526 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=86,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:42:02.918885 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27588f-0a56-46e9-80f0-08cb04b361cc None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:02.919529 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27588f-0a56-46e9-80f0-08cb04b361cc None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:02.919529 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27588f-0a56-46e9-80f0-08cb04b361cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:02.938881 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5b27588f-0a56-46e9-80f0-08cb04b361cc None None] MFA Rules not processed for user `66ad8defb9b6453c96f1a8a7343695ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:42:02.940119 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5b27588f-0a56-46e9-80f0-08cb04b361cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:42:02.947233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1340/2675] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jun 30 10:42:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:42:02.970179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda7d8c-49a6-4614-81e8-14334755e2be service placement] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:02.981499 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bda7d8c-49a6-4614-81e8-14334755e2be service placement] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:02.987318 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d49c95-5226-4e07-81b7-b335e8c62e24 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07de926cb8594b098f79111dab8a9cc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:02.988019 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d49c95-5226-4e07-81b7-b335e8c62e24 service swift] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:02.988331 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d49c95-5226-4e07-81b7-b335e8c62e24 service swift] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:02.988639 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d49c95-5226-4e07-81b7-b335e8c62e24 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:02.997017 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d49c95-5226-4e07-81b7-b335e8c62e24 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:02.998588 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d49c95-5226-4e07-81b7-b335e8c62e24 service swift] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:03.006977 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1336/2676] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jun 30 10:42:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:42:05.804295 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27588f-0a56-46e9-80f0-08cb04b361cc None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:05.805873 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5b27588f-0a56-46e9-80f0-08cb04b361cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:42:05.898126 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b27588f-0a56-46e9-80f0-08cb04b361cc None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:05.903888 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c630efaa-89b5-4f15-a104-4889c2f3fadf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:05.904580 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630efaa-89b5-4f15-a104-4889c2f3fadf service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:05.904869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630efaa-89b5-4f15-a104-4889c2f3fadf service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:05.905167 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c630efaa-89b5-4f15-a104-4889c2f3fadf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:05.907348 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c630efaa-89b5-4f15-a104-4889c2f3fadf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:42:05.977794 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c630efaa-89b5-4f15-a104-4889c2f3fadf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:05.979474 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c630efaa-89b5-4f15-a104-4889c2f3fadf service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:06.041973 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1341/2677] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:42:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:42:13.004841 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=63 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:42:13.191798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d49c95-5226-4e07-81b7-b335e8c62e24 service swift] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:13.196014 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-74d49c95-5226-4e07-81b7-b335e8c62e24 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:42:13.312274 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d49c95-5226-4e07-81b7-b335e8c62e24 service swift] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:13.323439 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1efabb-f3ed-4579-a086-0b904672d7e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:13.325888 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1efabb-f3ed-4579-a086-0b904672d7e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:13.327544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1efabb-f3ed-4579-a086-0b904672d7e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:13.329202 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1efabb-f3ed-4579-a086-0b904672d7e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:13.334751 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-6e1efabb-f3ed-4579-a086-0b904672d7e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:42:13.428931 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1efabb-f3ed-4579-a086-0b904672d7e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:13.431141 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1efabb-f3ed-4579-a086-0b904672d7e0 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:13.455492 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1337/2678] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:42:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:42:15.984908 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=59 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:42:21.783661 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a7bf4-2842-4e03-90f9-895e459d5db7 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:21.784274 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a7bf4-2842-4e03-90f9-895e459d5db7 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:21.784274 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0a7bf4-2842-4e03-90f9-895e459d5db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:21.803756 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-0b0a7bf4-2842-4e03-90f9-895e459d5db7 None None] MFA Rules not processed for user `66ad8defb9b6453c96f1a8a7343695ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:42:21.805065 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0b0a7bf4-2842-4e03-90f9-895e459d5db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:42:21.813003 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1342/2679] 10.222.0.47 () {58 vars in 967 bytes} [Thu Jun 30 10:42:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:42:21.833498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1efabb-f3ed-4579-a086-0b904672d7e0 service neutron] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:21.847999 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1efabb-f3ed-4579-a086-0b904672d7e0 service neutron] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:21.853289 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebc502-7315-4a2c-99a7-dfa060cf46a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07de926cb8594b098f79111dab8a9cc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:21.854016 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebc502-7315-4a2c-99a7-dfa060cf46a7 service swift] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:21.854392 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebc502-7315-4a2c-99a7-dfa060cf46a7 service swift] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:21.854738 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebc502-7315-4a2c-99a7-dfa060cf46a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:21.864519 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ebc502-7315-4a2c-99a7-dfa060cf46a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:21.866645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ebc502-7315-4a2c-99a7-dfa060cf46a7 service swift] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:21.874883 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1338/2680] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jun 30 10:42:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 30 10:42:22.278276 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a7bf4-2842-4e03-90f9-895e459d5db7 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:22.280524 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0b0a7bf4-2842-4e03-90f9-895e459d5db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:42:22.295696 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0a7bf4-2842-4e03-90f9-895e459d5db7 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:22.301444 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fef7b9-ac1e-48bd-beb9-4d481e5ca298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:22.302315 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fef7b9-ac1e-48bd-beb9-4d481e5ca298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:22.302596 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fef7b9-ac1e-48bd-beb9-4d481e5ca298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:22.302868 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fef7b9-ac1e-48bd-beb9-4d481e5ca298 admin admin] PATH_INFO: `/v3/users/1b151de186ff48b193672ec104e031a1` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:22.304093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fef7b9-ac1e-48bd-beb9-4d481e5ca298 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b151de186ff48b193672ec104e031a1)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:22.305997 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fef7b9-ac1e-48bd-beb9-4d481e5ca298 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:22.599114 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c5fef7b9-ac1e-48bd-beb9-4d481e5ca298 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b151de186ff48b193672ec104e031a1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:22.599890 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c5fef7b9-ac1e-48bd-beb9-4d481e5ca298 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b151de186ff48b193672ec104e031a1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:22.611354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c5fef7b9-ac1e-48bd-beb9-4d481e5ca298 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b151de186ff48b193672ec104e031a1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:22.612429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c5fef7b9-ac1e-48bd-beb9-4d481e5ca298 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b151de186ff48b193672ec104e031a1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:22.747431 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c5fef7b9-ac1e-48bd-beb9-4d481e5ca298 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b151de186ff48b193672ec104e031a1'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:22.777251 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1343/2681] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:42:22 2022] DELETE /identity/v3/users/1b151de186ff48b193672ec104e031a1 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:42:23.406631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebc502-7315-4a2c-99a7-dfa060cf46a7 service swift] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:23.418331 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ebc502-7315-4a2c-99a7-dfa060cf46a7 service swift] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:23.484516 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492c1cf-d835-4d7e-9358-20afee9bb6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:23.485401 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492c1cf-d835-4d7e-9358-20afee9bb6f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:23.485689 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492c1cf-d835-4d7e-9358-20afee9bb6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:23.485968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492c1cf-d835-4d7e-9358-20afee9bb6f4 admin admin] PATH_INFO: `/v3/projects/3a35109cec4141168620f42478a95c9c` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:23.487204 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3492c1cf-d835-4d7e-9358-20afee9bb6f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a35109cec4141168620f42478a95c9c)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:23.489182 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3492c1cf-d835-4d7e-9358-20afee9bb6f4 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:23.769383 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:42:23.778508 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-3492c1cf-d835-4d7e-9358-20afee9bb6f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a35109cec4141168620f42478a95c9c', 'request_id': 'req-3492c1cf-d835-4d7e-9358-20afee9bb6f4'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:23.784755 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-3492c1cf-d835-4d7e-9358-20afee9bb6f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a35109cec4141168620f42478a95c9c', 'request_id': 'req-3492c1cf-d835-4d7e-9358-20afee9bb6f4'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:23.792128 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-3492c1cf-d835-4d7e-9358-20afee9bb6f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a35109cec4141168620f42478a95c9c', 'request_id': 'req-3492c1cf-d835-4d7e-9358-20afee9bb6f4'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:23.919161 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-3492c1cf-d835-4d7e-9358-20afee9bb6f4 admin admin] The token cache is being invalidate because project 3a35109cec4141168620f42478a95c9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:42:23.919161 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-3492c1cf-d835-4d7e-9358-20afee9bb6f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:23.919787 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-3492c1cf-d835-4d7e-9358-20afee9bb6f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:23.921471 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1339/2682] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:42:23 2022] DELETE /identity/v3/projects/3a35109cec4141168620f42478a95c9c => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:42:23.925540 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fef7b9-ac1e-48bd-beb9-4d481e5ca298 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:23.927022 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c5fef7b9-ac1e-48bd-beb9-4d481e5ca298 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:42:23.941207 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fef7b9-ac1e-48bd-beb9-4d481e5ca298 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:24.000859 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb20ce-27c2-4032-b57e-4b572a7c94e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:24.001765 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb20ce-27c2-4032-b57e-4b572a7c94e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:24.002036 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb20ce-27c2-4032-b57e-4b572a7c94e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:24.002316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb20ce-27c2-4032-b57e-4b572a7c94e8 admin admin] PATH_INFO: `/v3/users/fd5adf3f3cda419c8cc1c82d2c20f361` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:24.003596 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb20ce-27c2-4032-b57e-4b572a7c94e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd5adf3f3cda419c8cc1c82d2c20f361)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:24.005543 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb20ce-27c2-4032-b57e-4b572a7c94e8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:24.409332 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e3cb20ce-27c2-4032-b57e-4b572a7c94e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd5adf3f3cda419c8cc1c82d2c20f361'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:24.409991 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e3cb20ce-27c2-4032-b57e-4b572a7c94e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd5adf3f3cda419c8cc1c82d2c20f361'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:24.417124 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e3cb20ce-27c2-4032-b57e-4b572a7c94e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd5adf3f3cda419c8cc1c82d2c20f361'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:24.417752 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e3cb20ce-27c2-4032-b57e-4b572a7c94e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd5adf3f3cda419c8cc1c82d2c20f361'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:24.558338 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e3cb20ce-27c2-4032-b57e-4b572a7c94e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd5adf3f3cda419c8cc1c82d2c20f361'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:24.563956 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:42:24.577629 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1344/2683] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:42:23 2022] DELETE /identity/v3/users/fd5adf3f3cda419c8cc1c82d2c20f361 => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:42:24.877402 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492c1cf-d835-4d7e-9358-20afee9bb6f4 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:24.878861 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-3492c1cf-d835-4d7e-9358-20afee9bb6f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:42:24.892848 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492c1cf-d835-4d7e-9358-20afee9bb6f4 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:24.955105 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d39c6-63c7-4eab-86d7-4a0391ce1883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:24.956010 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d39c6-63c7-4eab-86d7-4a0391ce1883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:24.956466 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d39c6-63c7-4eab-86d7-4a0391ce1883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:24.956756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887d39c6-63c7-4eab-86d7-4a0391ce1883 admin admin] PATH_INFO: `/v3/projects/3a5f173a69c742e4aba26537e5b25d52` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:24.957956 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d39c6-63c7-4eab-86d7-4a0391ce1883 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a5f173a69c742e4aba26537e5b25d52)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:24.959928 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887d39c6-63c7-4eab-86d7-4a0391ce1883 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:25.118267 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-887d39c6-63c7-4eab-86d7-4a0391ce1883 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a5f173a69c742e4aba26537e5b25d52', 'request_id': 'req-887d39c6-63c7-4eab-86d7-4a0391ce1883'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:25.125048 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-887d39c6-63c7-4eab-86d7-4a0391ce1883 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a5f173a69c742e4aba26537e5b25d52', 'request_id': 'req-887d39c6-63c7-4eab-86d7-4a0391ce1883'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:25.130544 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-887d39c6-63c7-4eab-86d7-4a0391ce1883 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a5f173a69c742e4aba26537e5b25d52', 'request_id': 'req-887d39c6-63c7-4eab-86d7-4a0391ce1883'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:25.321997 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-887d39c6-63c7-4eab-86d7-4a0391ce1883 admin admin] The token cache is being invalidate because project 3a5f173a69c742e4aba26537e5b25d52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:42:25.322412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-887d39c6-63c7-4eab-86d7-4a0391ce1883 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:25.323109 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-887d39c6-63c7-4eab-86d7-4a0391ce1883 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:25.324739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1340/2684] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:42:24 2022] DELETE /identity/v3/projects/3a5f173a69c742e4aba26537e5b25d52 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:42:25.855516 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb20ce-27c2-4032-b57e-4b572a7c94e8 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:25.858786 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e3cb20ce-27c2-4032-b57e-4b572a7c94e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:42:25.878732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb20ce-27c2-4032-b57e-4b572a7c94e8 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:25.935520 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a55f45-3b15-4e13-90bb-0a7c48d7f319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:25.936908 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a55f45-3b15-4e13-90bb-0a7c48d7f319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:25.937197 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a55f45-3b15-4e13-90bb-0a7c48d7f319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:25.937475 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a55f45-3b15-4e13-90bb-0a7c48d7f319 admin admin] PATH_INFO: `/v3/users/e9cf01f5974c45c6bd19b3e51b23d89c` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:25.938682 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a55f45-3b15-4e13-90bb-0a7c48d7f319 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9cf01f5974c45c6bd19b3e51b23d89c)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:25.940696 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a55f45-3b15-4e13-90bb-0a7c48d7f319 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:26.152903 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-87a55f45-3b15-4e13-90bb-0a7c48d7f319 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9cf01f5974c45c6bd19b3e51b23d89c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:26.154464 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-87a55f45-3b15-4e13-90bb-0a7c48d7f319 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9cf01f5974c45c6bd19b3e51b23d89c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:26.167598 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-87a55f45-3b15-4e13-90bb-0a7c48d7f319 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9cf01f5974c45c6bd19b3e51b23d89c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:26.168495 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-87a55f45-3b15-4e13-90bb-0a7c48d7f319 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9cf01f5974c45c6bd19b3e51b23d89c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:26.253050 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-87a55f45-3b15-4e13-90bb-0a7c48d7f319 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9cf01f5974c45c6bd19b3e51b23d89c'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:26.272080 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1345/2685] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:42:25 2022] DELETE /identity/v3/users/e9cf01f5974c45c6bd19b3e51b23d89c => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:42:26.540983 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d39c6-63c7-4eab-86d7-4a0391ce1883 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:26.542844 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-887d39c6-63c7-4eab-86d7-4a0391ce1883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:42:26.561089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887d39c6-63c7-4eab-86d7-4a0391ce1883 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:26.631653 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:42:26.637811 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee82117d-da83-46be-97ef-fa7871a63e9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:26.638304 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82117d-da83-46be-97ef-fa7871a63e9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:26.638588 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82117d-da83-46be-97ef-fa7871a63e9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:26.639459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee82117d-da83-46be-97ef-fa7871a63e9a admin admin] PATH_INFO: `/v3/projects/5c62eb86ac194a7fa67c670fd6239743` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:26.641207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82117d-da83-46be-97ef-fa7871a63e9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c62eb86ac194a7fa67c670fd6239743)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:26.643159 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee82117d-da83-46be-97ef-fa7871a63e9a admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:26.782447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-ee82117d-da83-46be-97ef-fa7871a63e9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c62eb86ac194a7fa67c670fd6239743', 'request_id': 'req-ee82117d-da83-46be-97ef-fa7871a63e9a'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:26.788734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-ee82117d-da83-46be-97ef-fa7871a63e9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c62eb86ac194a7fa67c670fd6239743', 'request_id': 'req-ee82117d-da83-46be-97ef-fa7871a63e9a'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:26.794644 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-ee82117d-da83-46be-97ef-fa7871a63e9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c62eb86ac194a7fa67c670fd6239743', 'request_id': 'req-ee82117d-da83-46be-97ef-fa7871a63e9a'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:26.924607 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-ee82117d-da83-46be-97ef-fa7871a63e9a admin admin] The token cache is being invalidate because project 5c62eb86ac194a7fa67c670fd6239743 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:42:26.924895 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-ee82117d-da83-46be-97ef-fa7871a63e9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:26.925512 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-ee82117d-da83-46be-97ef-fa7871a63e9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:26.927059 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1341/2686] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:42:26 2022] DELETE /identity/v3/projects/5c62eb86ac194a7fa67c670fd6239743 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:42:26.932075 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa6ec6-3153-46be-85b0-f48f163853b2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:26.932566 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa6ec6-3153-46be-85b0-f48f163853b2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:26.932664 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa6ec6-3153-46be-85b0-f48f163853b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:26.952128 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-defa6ec6-3153-46be-85b0-f48f163853b2 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:42:26.953350 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-defa6ec6-3153-46be-85b0-f48f163853b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:42:27.011991 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1346/2687] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:42:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:42:27.020967 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25808164-0f26-4cbf-a40b-78b650e6c4bf None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:27.021257 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25808164-0f26-4cbf-a40b-78b650e6c4bf None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:27.025668 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25808164-0f26-4cbf-a40b-78b650e6c4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:27.039900 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-25808164-0f26-4cbf-a40b-78b650e6c4bf None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:42:27.041112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-25808164-0f26-4cbf-a40b-78b650e6c4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:42:27.049244 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1342/2688] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:42:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:42:27.053601 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa6ec6-3153-46be-85b0-f48f163853b2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:27.059703 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:42:27.064452 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa6ec6-3153-46be-85b0-f48f163853b2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:27.069032 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5485b8-f707-47c5-b61a-6e737ec8281a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:27.069677 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5485b8-f707-47c5-b61a-6e737ec8281a admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:27.069983 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5485b8-f707-47c5-b61a-6e737ec8281a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:27.071661 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5485b8-f707-47c5-b61a-6e737ec8281a admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:27.071661 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5485b8-f707-47c5-b61a-6e737ec8281a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:42:27.071661 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5485b8-f707-47c5-b61a-6e737ec8281a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:27.072982 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5485b8-f707-47c5-b61a-6e737ec8281a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:27.085578 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1347/2689] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:42:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:42:27.089717 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25808164-0f26-4cbf-a40b-78b650e6c4bf None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:27.099539 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25808164-0f26-4cbf-a40b-78b650e6c4bf None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:27.104204 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16084bce-bf12-46c6-bc0f-fe4695f55fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:27.106075 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16084bce-bf12-46c6-bc0f-fe4695f55fb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:27.106075 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16084bce-bf12-46c6-bc0f-fe4695f55fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:27.106075 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16084bce-bf12-46c6-bc0f-fe4695f55fb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:27.106075 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16084bce-bf12-46c6-bc0f-fe4695f55fb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:27.109175 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16084bce-bf12-46c6-bc0f-fe4695f55fb5 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:27.194819 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1343/2690] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:42:27 2022] POST /identity/v3/projects => generated 371 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:42:27.199414 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5485b8-f707-47c5-b61a-6e737ec8281a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:27.209594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5485b8-f707-47c5-b61a-6e737ec8281a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:27.267108 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062401f-b64b-469c-b08c-44366ffd282d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:27.267720 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062401f-b64b-469c-b08c-44366ffd282d admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:27.268001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062401f-b64b-469c-b08c-44366ffd282d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:27.268275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062401f-b64b-469c-b08c-44366ffd282d admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:27.268957 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062401f-b64b-469c-b08c-44366ffd282d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:27.270885 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062401f-b64b-469c-b08c-44366ffd282d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:27.353042 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1348/2691] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:42:27 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:42:27.357240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16084bce-bf12-46c6-bc0f-fe4695f55fb5 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:27.368126 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16084bce-bf12-46c6-bc0f-fe4695f55fb5 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:27.372784 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b3d4f-1603-4a59-bd23-1f8c9679986c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:27.374119 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b3d4f-1603-4a59-bd23-1f8c9679986c admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:27.374540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b3d4f-1603-4a59-bd23-1f8c9679986c admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:27.374824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4b3d4f-1603-4a59-bd23-1f8c9679986c admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:27.375592 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b3d4f-1603-4a59-bd23-1f8c9679986c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:27.377231 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4b3d4f-1603-4a59-bd23-1f8c9679986c admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:27.387438 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1344/2692] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:42:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:42:27.391591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062401f-b64b-469c-b08c-44366ffd282d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:27.401637 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062401f-b64b-469c-b08c-44366ffd282d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:27.406685 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed610d1-0116-49b8-a260-9db76e458748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:27.407363 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed610d1-0116-49b8-a260-9db76e458748 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:27.407645 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed610d1-0116-49b8-a260-9db76e458748 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:27.407920 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed610d1-0116-49b8-a260-9db76e458748 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:27.408638 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed610d1-0116-49b8-a260-9db76e458748 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:27.410223 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed610d1-0116-49b8-a260-9db76e458748 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:27.420921 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1349/2693] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:42:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:42:27.424828 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b3d4f-1603-4a59-bd23-1f8c9679986c admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:27.434577 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4b3d4f-1603-4a59-bd23-1f8c9679986c admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:27.439797 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2b9e1-daed-4812-97db-2037dfd6afcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:27.440667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2b9e1-daed-4812-97db-2037dfd6afcb admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:27.440963 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2b9e1-daed-4812-97db-2037dfd6afcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:27.441241 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a2b9e1-daed-4812-97db-2037dfd6afcb admin admin] PATH_INFO: `/v3/projects/31a8483f987846bc91c434eb1f5af2e3/users/895e6b7ecb494e2dae09e1f991178590/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:27.456757 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a2b9e1-daed-4812-97db-2037dfd6afcb admin admin] RBAC: Authorizing `identity:create_grant(project_id=31a8483f987846bc91c434eb1f5af2e3, user_id=895e6b7ecb494e2dae09e1f991178590, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:27.459433 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a2b9e1-daed-4812-97db-2037dfd6afcb admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:27.534312 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1345/2694] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:42:27 2022] PUT /identity/v3/projects/31a8483f987846bc91c434eb1f5af2e3/users/895e6b7ecb494e2dae09e1f991178590/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:42:27.544691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7032a78-c81c-4bb2-a97a-52f5ef2e4a98 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:27.544988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7032a78-c81c-4bb2-a97a-52f5ef2e4a98 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:27.545951 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7032a78-c81c-4bb2-a97a-52f5ef2e4a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:27.572763 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c7032a78-c81c-4bb2-a97a-52f5ef2e4a98 None None] MFA Rules not processed for user `895e6b7ecb494e2dae09e1f991178590`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:42:27.573980 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c7032a78-c81c-4bb2-a97a-52f5ef2e4a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:42:27.635994 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1350/2695] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:42:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:42:27.647349 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2b9e1-daed-4812-97db-2037dfd6afcb admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:27.649017 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-70a2b9e1-daed-4812-97db-2037dfd6afcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:42:27.707337 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a2b9e1-daed-4812-97db-2037dfd6afcb admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:27.711626 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dccc96-aedb-4e2a-a4a0-6c235e7e7982 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c7a61d85908d42698b27b9fc00daa832', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:27.712280 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dccc96-aedb-4e2a-a4a0-6c235e7e7982 service glance] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:27.712569 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dccc96-aedb-4e2a-a4a0-6c235e7e7982 service glance] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:27.712851 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dccc96-aedb-4e2a-a4a0-6c235e7e7982 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:27.722810 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dccc96-aedb-4e2a-a4a0-6c235e7e7982 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:27.724282 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dccc96-aedb-4e2a-a4a0-6c235e7e7982 service glance] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:27.734295 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1346/2696] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:42:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:42:33.826143 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8603d-5842-4758-a35f-380828bcafb8 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:33.830594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8603d-5842-4758-a35f-380828bcafb8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:33.830594 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe8603d-5842-4758-a35f-380828bcafb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:33.845149 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-abe8603d-5842-4758-a35f-380828bcafb8 None None] MFA Rules not processed for user `1f4a95c82152441096c50f79acc68e9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:42:33.846415 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-abe8603d-5842-4758-a35f-380828bcafb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:42:33.907999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1351/2697] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:42:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:42:37.729141 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:42:39.074022 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01eb80-d39a-4258-8a29-8725d71d9370 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:39.074022 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01eb80-d39a-4258-8a29-8725d71d9370 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:39.074022 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01eb80-d39a-4258-8a29-8725d71d9370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:39.118692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-ea01eb80-d39a-4258-8a29-8725d71d9370 None None] MFA Rules not processed for user `fa58de3ae9eb4dc9bd80fc41ec3e0793`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:42:39.122221 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-ea01eb80-d39a-4258-8a29-8725d71d9370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:42:39.197240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1347/2698] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:42:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:42:43.900381 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=95,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:42:49.184827 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=25 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:42:56.652415 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8603d-5842-4758-a35f-380828bcafb8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:56.715221 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe8603d-5842-4758-a35f-380828bcafb8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:56.720658 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897fd7a-47d4-4723-b0a7-0f7e30cbaf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:56.721530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897fd7a-47d4-4723-b0a7-0f7e30cbaf09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:56.721812 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897fd7a-47d4-4723-b0a7-0f7e30cbaf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:56.722085 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e897fd7a-47d4-4723-b0a7-0f7e30cbaf09 admin admin] PATH_INFO: `/v3/users/895e6b7ecb494e2dae09e1f991178590` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:56.723303 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897fd7a-47d4-4723-b0a7-0f7e30cbaf09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=895e6b7ecb494e2dae09e1f991178590)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:56.725659 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e897fd7a-47d4-4723-b0a7-0f7e30cbaf09 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:56.939641 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e897fd7a-47d4-4723-b0a7-0f7e30cbaf09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '895e6b7ecb494e2dae09e1f991178590'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:56.940297 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e897fd7a-47d4-4723-b0a7-0f7e30cbaf09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '895e6b7ecb494e2dae09e1f991178590'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:56.947684 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e897fd7a-47d4-4723-b0a7-0f7e30cbaf09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '895e6b7ecb494e2dae09e1f991178590'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:56.948312 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e897fd7a-47d4-4723-b0a7-0f7e30cbaf09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '895e6b7ecb494e2dae09e1f991178590'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:57.012094 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e897fd7a-47d4-4723-b0a7-0f7e30cbaf09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '895e6b7ecb494e2dae09e1f991178590'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:57.030788 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1352/2699] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:42:56 2022] DELETE /identity/v3/users/895e6b7ecb494e2dae09e1f991178590 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:42:57.035382 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a5b73-c984-484f-a68c-826e9e425e9f None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:57.035888 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a5b73-c984-484f-a68c-826e9e425e9f None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:57.035990 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a5b73-c984-484f-a68c-826e9e425e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:57.054031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-967a5b73-c984-484f-a68c-826e9e425e9f None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:42:57.057047 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-967a5b73-c984-484f-a68c-826e9e425e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:42:57.118775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1348/2700] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:42:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:42:57.126889 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897fd7a-47d4-4723-b0a7-0f7e30cbaf09 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:57.128347 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e897fd7a-47d4-4723-b0a7-0f7e30cbaf09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:42:57.189522 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e897fd7a-47d4-4723-b0a7-0f7e30cbaf09 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:57.193051 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f6dd27-d51e-4805-b181-ed265f4fb5d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:57.193702 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6dd27-d51e-4805-b181-ed265f4fb5d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:57.193984 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6dd27-d51e-4805-b181-ed265f4fb5d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:57.194259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f6dd27-d51e-4805-b181-ed265f4fb5d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:57.204044 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f6dd27-d51e-4805-b181-ed265f4fb5d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:57.205592 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f6dd27-d51e-4805-b181-ed265f4fb5d4 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:57.215412 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1353/2701] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:42:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:42:57.749666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a5b73-c984-484f-a68c-826e9e425e9f None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:57.751090 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-967a5b73-c984-484f-a68c-826e9e425e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:42:57.762573 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a5b73-c984-484f-a68c-826e9e425e9f None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:57.768035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e30e27-4bae-4327-b328-0abe1d51c4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:57.769524 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e30e27-4bae-4327-b328-0abe1d51c4cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:57.769524 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e30e27-4bae-4327-b328-0abe1d51c4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:57.769524 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e30e27-4bae-4327-b328-0abe1d51c4cc admin admin] PATH_INFO: `/v3/projects/31a8483f987846bc91c434eb1f5af2e3` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:57.770665 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e30e27-4bae-4327-b328-0abe1d51c4cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=31a8483f987846bc91c434eb1f5af2e3)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:57.772623 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e30e27-4bae-4327-b328-0abe1d51c4cc admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:57.963177 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-13e30e27-4bae-4327-b328-0abe1d51c4cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31a8483f987846bc91c434eb1f5af2e3', 'request_id': 'req-13e30e27-4bae-4327-b328-0abe1d51c4cc'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:57.970700 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-13e30e27-4bae-4327-b328-0abe1d51c4cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31a8483f987846bc91c434eb1f5af2e3', 'request_id': 'req-13e30e27-4bae-4327-b328-0abe1d51c4cc'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:57.977070 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-13e30e27-4bae-4327-b328-0abe1d51c4cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31a8483f987846bc91c434eb1f5af2e3', 'request_id': 'req-13e30e27-4bae-4327-b328-0abe1d51c4cc'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:58.128031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-13e30e27-4bae-4327-b328-0abe1d51c4cc admin admin] The token cache is being invalidate because project 31a8483f987846bc91c434eb1f5af2e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:42:58.128688 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-13e30e27-4bae-4327-b328-0abe1d51c4cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:58.133942 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-13e30e27-4bae-4327-b328-0abe1d51c4cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:42:58.133942 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1349/2702] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:42:57 2022] DELETE /identity/v3/projects/31a8483f987846bc91c434eb1f5af2e3 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:42:58.135460 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a3a82-0eb1-4510-a022-8a9108ce48c7 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:58.135939 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a3a82-0eb1-4510-a022-8a9108ce48c7 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:58.136186 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7a3a82-0eb1-4510-a022-8a9108ce48c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:58.155642 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-1c7a3a82-0eb1-4510-a022-8a9108ce48c7 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:42:58.156957 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-1c7a3a82-0eb1-4510-a022-8a9108ce48c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:42:58.175400 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:42:58.219704 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1354/2703] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:42:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:42:58.227929 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a4c9d-69f6-4973-bbee-23b3bf11072e None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:58.228219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a4c9d-69f6-4973-bbee-23b3bf11072e None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:58.228534 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844a4c9d-69f6-4973-bbee-23b3bf11072e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:58.247181 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-844a4c9d-69f6-4973-bbee-23b3bf11072e None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:42:58.248911 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-844a4c9d-69f6-4973-bbee-23b3bf11072e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:42:58.258889 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1350/2704] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:42:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:42:58.263043 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a3a82-0eb1-4510-a022-8a9108ce48c7 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:58.273578 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7a3a82-0eb1-4510-a022-8a9108ce48c7 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:58.278306 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3dce00-f0d7-4d52-87c4-83be0d8f21b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:58.278960 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3dce00-f0d7-4d52-87c4-83be0d8f21b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:58.279245 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3dce00-f0d7-4d52-87c4-83be0d8f21b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:58.279591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3dce00-f0d7-4d52-87c4-83be0d8f21b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:58.280483 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3dce00-f0d7-4d52-87c4-83be0d8f21b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:42:58.280794 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3dce00-f0d7-4d52-87c4-83be0d8f21b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:58.282402 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3dce00-f0d7-4d52-87c4-83be0d8f21b5 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:58.294371 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1355/2705] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:42:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:42:58.298410 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a4c9d-69f6-4973-bbee-23b3bf11072e None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:58.307996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844a4c9d-69f6-4973-bbee-23b3bf11072e None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:58.313435 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e618f-eceb-4330-83b9-f23213088413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:58.314096 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e618f-eceb-4330-83b9-f23213088413 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:58.314384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e618f-eceb-4330-83b9-f23213088413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:58.314663 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354e618f-eceb-4330-83b9-f23213088413 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:58.315399 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e618f-eceb-4330-83b9-f23213088413 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:58.317436 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354e618f-eceb-4330-83b9-f23213088413 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:58.460589 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1351/2706] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:42:58 2022] POST /identity/v3/projects => generated 399 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:42:58.464986 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3dce00-f0d7-4d52-87c4-83be0d8f21b5 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:58.475348 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3dce00-f0d7-4d52-87c4-83be0d8f21b5 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:58.532682 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79456ad-edff-4d63-8b6c-1576881d48ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:58.533357 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79456ad-edff-4d63-8b6c-1576881d48ca admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:58.533639 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79456ad-edff-4d63-8b6c-1576881d48ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:58.533915 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79456ad-edff-4d63-8b6c-1576881d48ca admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:58.534615 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79456ad-edff-4d63-8b6c-1576881d48ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:58.536839 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79456ad-edff-4d63-8b6c-1576881d48ca admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:58.709780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1356/2707] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:42:58 2022] POST /identity/v3/users => generated 300 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:42:58.713831 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e618f-eceb-4330-83b9-f23213088413 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:58.724357 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354e618f-eceb-4330-83b9-f23213088413 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:58.729085 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e437f49-ac07-4446-9789-1d044236f629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:58.729688 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e437f49-ac07-4446-9789-1d044236f629 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:58.729975 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e437f49-ac07-4446-9789-1d044236f629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:58.730254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e437f49-ac07-4446-9789-1d044236f629 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:58.730980 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e437f49-ac07-4446-9789-1d044236f629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:58.733131 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e437f49-ac07-4446-9789-1d044236f629 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:58.743734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1352/2708] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:42:58.747703 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79456ad-edff-4d63-8b6c-1576881d48ca admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:58.758263 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79456ad-edff-4d63-8b6c-1576881d48ca admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:58.762813 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e9fda-e4e1-4522-bda8-1e347fd8cf59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:58.763512 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e9fda-e4e1-4522-bda8-1e347fd8cf59 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:58.763799 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e9fda-e4e1-4522-bda8-1e347fd8cf59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:58.764206 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e9fda-e4e1-4522-bda8-1e347fd8cf59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:58.765039 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e9fda-e4e1-4522-bda8-1e347fd8cf59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:58.766656 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e9fda-e4e1-4522-bda8-1e347fd8cf59 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:58.776551 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1357/2709] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:42:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:42:58.780652 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e437f49-ac07-4446-9789-1d044236f629 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:58.790958 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e437f49-ac07-4446-9789-1d044236f629 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:58.796352 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce02d3a-40bf-451e-a749-5039cc11fd11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:58.797384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce02d3a-40bf-451e-a749-5039cc11fd11 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:58.797673 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce02d3a-40bf-451e-a749-5039cc11fd11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:58.797950 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce02d3a-40bf-451e-a749-5039cc11fd11 admin admin] PATH_INFO: `/v3/projects/aac1d8bee0d04cafa2d821016330220e/users/c89f3a8ba3b34bf09a00c5461c289a99/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:58.813237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce02d3a-40bf-451e-a749-5039cc11fd11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aac1d8bee0d04cafa2d821016330220e, user_id=c89f3a8ba3b34bf09a00c5461c289a99, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:58.815956 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce02d3a-40bf-451e-a749-5039cc11fd11 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:58.882007 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1353/2710] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:42:58 2022] PUT /identity/v3/projects/aac1d8bee0d04cafa2d821016330220e/users/c89f3a8ba3b34bf09a00c5461c289a99/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:42:58.893463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cb6f2-1192-4b1e-9040-a887cb5fcb3d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:58.893753 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cb6f2-1192-4b1e-9040-a887cb5fcb3d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:58.894035 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cb6f2-1192-4b1e-9040-a887cb5fcb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:58.921024 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d24cb6f2-1192-4b1e-9040-a887cb5fcb3d None None] MFA Rules not processed for user `c89f3a8ba3b34bf09a00c5461c289a99`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:42:58.922244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d24cb6f2-1192-4b1e-9040-a887cb5fcb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:42:58.988024 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1358/2711] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:42:58 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:42:59.003192 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce02d3a-40bf-451e-a749-5039cc11fd11 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:42:59.003192 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-dce02d3a-40bf-451e-a749-5039cc11fd11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:42:59.024767 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:42:59.060518 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce02d3a-40bf-451e-a749-5039cc11fd11 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:42:59.064691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b7844-c15e-43df-9025-e3d0e9643b9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:42:59.065335 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b7844-c15e-43df-9025-e3d0e9643b9e service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:42:59.065617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b7844-c15e-43df-9025-e3d0e9643b9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:42:59.065892 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b7844-c15e-43df-9025-e3d0e9643b9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:42:59.074036 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b7844-c15e-43df-9025-e3d0e9643b9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:42:59.075486 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845b7844-c15e-43df-9025-e3d0e9643b9e service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:42:59.084919 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1354/2712] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:42:58 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:43:06.989992 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24cb6f2-1192-4b1e-9040-a887cb5fcb3d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:43:07.009206 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:43:07.051866 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24cb6f2-1192-4b1e-9040-a887cb5fcb3d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:43:07.056383 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e741c22-ca91-4c60-ac9a-5e4aad8caadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:43:07.057253 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e741c22-ca91-4c60-ac9a-5e4aad8caadb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:43:07.057537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e741c22-ca91-4c60-ac9a-5e4aad8caadb admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:43:07.057816 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e741c22-ca91-4c60-ac9a-5e4aad8caadb admin admin] PATH_INFO: `/v3/users/c89f3a8ba3b34bf09a00c5461c289a99` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:43:07.058999 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e741c22-ca91-4c60-ac9a-5e4aad8caadb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c89f3a8ba3b34bf09a00c5461c289a99)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:43:07.061069 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e741c22-ca91-4c60-ac9a-5e4aad8caadb admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:43:07.294913 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0e741c22-ca91-4c60-ac9a-5e4aad8caadb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c89f3a8ba3b34bf09a00c5461c289a99'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:43:07.295797 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0e741c22-ca91-4c60-ac9a-5e4aad8caadb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c89f3a8ba3b34bf09a00c5461c289a99'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:43:07.302979 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0e741c22-ca91-4c60-ac9a-5e4aad8caadb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c89f3a8ba3b34bf09a00c5461c289a99'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:43:07.303610 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0e741c22-ca91-4c60-ac9a-5e4aad8caadb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c89f3a8ba3b34bf09a00c5461c289a99'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:43:07.369181 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-0e741c22-ca91-4c60-ac9a-5e4aad8caadb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c89f3a8ba3b34bf09a00c5461c289a99'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:43:07.388001 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1359/2713] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:43:06 2022] DELETE /identity/v3/users/c89f3a8ba3b34bf09a00c5461c289a99 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:43:07.393139 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a34367-84b9-447f-82a1-659c696b7402 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:43:07.393659 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a34367-84b9-447f-82a1-659c696b7402 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:43:07.393934 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a34367-84b9-447f-82a1-659c696b7402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:43:07.411941 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-23a34367-84b9-447f-82a1-659c696b7402 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:43:07.413508 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-23a34367-84b9-447f-82a1-659c696b7402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:43:07.473157 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1355/2714] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:43:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:43:07.484382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e741c22-ca91-4c60-ac9a-5e4aad8caadb admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:43:07.486046 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-0e741c22-ca91-4c60-ac9a-5e4aad8caadb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:43:07.564966 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e741c22-ca91-4c60-ac9a-5e4aad8caadb admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:43:07.568666 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e41869-03bf-4546-9caa-495486947187 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:43:07.569339 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e41869-03bf-4546-9caa-495486947187 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:43:07.569628 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e41869-03bf-4546-9caa-495486947187 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:43:07.569901 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e41869-03bf-4546-9caa-495486947187 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:43:07.580686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e41869-03bf-4546-9caa-495486947187 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:43:07.582163 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e41869-03bf-4546-9caa-495486947187 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:43:07.593191 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1360/2715] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:43:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:43:08.034223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a34367-84b9-447f-82a1-659c696b7402 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:43:08.040142 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-23a34367-84b9-447f-82a1-659c696b7402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:43:08.050089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a34367-84b9-447f-82a1-659c696b7402 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:43:08.055823 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48f504-c005-439e-b63f-3e8552e089d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:43:08.056782 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48f504-c005-439e-b63f-3e8552e089d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:43:08.057137 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48f504-c005-439e-b63f-3e8552e089d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:43:08.057489 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48f504-c005-439e-b63f-3e8552e089d8 admin admin] PATH_INFO: `/v3/projects/aac1d8bee0d04cafa2d821016330220e` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:43:08.058769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48f504-c005-439e-b63f-3e8552e089d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aac1d8bee0d04cafa2d821016330220e)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:43:08.060853 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48f504-c005-439e-b63f-3e8552e089d8 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:43:08.206654 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-6f48f504-c005-439e-b63f-3e8552e089d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aac1d8bee0d04cafa2d821016330220e', 'request_id': 'req-6f48f504-c005-439e-b63f-3e8552e089d8'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:43:08.213040 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-6f48f504-c005-439e-b63f-3e8552e089d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aac1d8bee0d04cafa2d821016330220e', 'request_id': 'req-6f48f504-c005-439e-b63f-3e8552e089d8'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:43:08.220706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-6f48f504-c005-439e-b63f-3e8552e089d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aac1d8bee0d04cafa2d821016330220e', 'request_id': 'req-6f48f504-c005-439e-b63f-3e8552e089d8'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:43:08.292056 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-6f48f504-c005-439e-b63f-3e8552e089d8 admin admin] The token cache is being invalidate because project aac1d8bee0d04cafa2d821016330220e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:43:08.299830 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-6f48f504-c005-439e-b63f-3e8552e089d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:43:08.299830 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-6f48f504-c005-439e-b63f-3e8552e089d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:43:08.299830 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1356/2716] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:43:08 2022] DELETE /identity/v3/projects/aac1d8bee0d04cafa2d821016330220e => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:43:17.588134 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:43:18.288693 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:43:44.997263 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e41869-03bf-4546-9caa-495486947187 service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:43:45.005929 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e41869-03bf-4546-9caa-495486947187 service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:43:45.053879 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ec905a-4e11-4d55-9eb6-8acdd9f4ae50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:43:45.054522 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec905a-4e11-4d55-9eb6-8acdd9f4ae50 service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:43:45.054805 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec905a-4e11-4d55-9eb6-8acdd9f4ae50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:43:45.055085 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ec905a-4e11-4d55-9eb6-8acdd9f4ae50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:43:45.056932 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e7ec905a-4e11-4d55-9eb6-8acdd9f4ae50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:43:45.126758 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ec905a-4e11-4d55-9eb6-8acdd9f4ae50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:43:45.128241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ec905a-4e11-4d55-9eb6-8acdd9f4ae50 service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:43:45.139421 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1361/2717] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:43:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:43:46.316831 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48f504-c005-439e-b63f-3e8552e089d8 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:43:46.318238 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-6f48f504-c005-439e-b63f-3e8552e089d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:43:46.330771 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48f504-c005-439e-b63f-3e8552e089d8 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:43:46.334405 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359fdf9-0907-434a-aaf1-325557ba0af1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:43:46.337975 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359fdf9-0907-434a-aaf1-325557ba0af1 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:43:46.337975 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359fdf9-0907-434a-aaf1-325557ba0af1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:43:46.337975 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359fdf9-0907-434a-aaf1-325557ba0af1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:43:46.337975 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-3359fdf9-0907-434a-aaf1-325557ba0af1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:43:46.352028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3359fdf9-0907-434a-aaf1-325557ba0af1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:43:46.353459 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3359fdf9-0907-434a-aaf1-325557ba0af1 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:43:46.364745 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1357/2718] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:43:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:43:48.844064 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5149b2-c3f6-494f-8d4f-08a2003b65c6 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:43:48.845140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5149b2-c3f6-494f-8d4f-08a2003b65c6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:43:48.845316 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5149b2-c3f6-494f-8d4f-08a2003b65c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:43:48.864455 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-db5149b2-c3f6-494f-8d4f-08a2003b65c6 None None] MFA Rules not processed for user `66ad8defb9b6453c96f1a8a7343695ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:43:48.865599 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-db5149b2-c3f6-494f-8d4f-08a2003b65c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:43:48.923204 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1362/2719] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jun 30 10:43:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:43:49.074797 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359fdf9-0907-434a-aaf1-325557ba0af1 service neutron] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:43:49.076225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-3359fdf9-0907-434a-aaf1-325557ba0af1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:43:49.149200 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359fdf9-0907-434a-aaf1-325557ba0af1 service neutron] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:43:49.154158 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7324c1-4460-4651-acf1-ca8d525c6a2e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07de926cb8594b098f79111dab8a9cc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:43:49.154807 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7324c1-4460-4651-acf1-ca8d525c6a2e service swift] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:43:49.155094 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7324c1-4460-4651-acf1-ca8d525c6a2e service swift] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:43:49.155471 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7324c1-4460-4651-acf1-ca8d525c6a2e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:43:49.164340 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7324c1-4460-4651-acf1-ca8d525c6a2e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:43:49.165912 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7324c1-4460-4651-acf1-ca8d525c6a2e service swift] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:43:49.173873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1358/2720] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jun 30 10:43:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:43:58.915867 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=79 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:43:59.170687 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=38 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:44:05.943976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5149b2-c3f6-494f-8d4f-08a2003b65c6 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:44:05.945394 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-db5149b2-c3f6-494f-8d4f-08a2003b65c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:44:06.029937 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5149b2-c3f6-494f-8d4f-08a2003b65c6 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:44:06.037233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb3d2a2-7eb7-420c-8d88-08b32abb1d2d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46c99f43a9344d7d92632990e45bb9cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:44:06.037890 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb3d2a2-7eb7-420c-8d88-08b32abb1d2d service placement] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:44:06.038430 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb3d2a2-7eb7-420c-8d88-08b32abb1d2d service placement] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:44:06.038715 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb3d2a2-7eb7-420c-8d88-08b32abb1d2d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:44:06.040512 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-dcb3d2a2-7eb7-420c-8d88-08b32abb1d2d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:44:06.052147 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb3d2a2-7eb7-420c-8d88-08b32abb1d2d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:44:06.053750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb3d2a2-7eb7-420c-8d88-08b32abb1d2d service placement] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:44:06.064166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1363/2721] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:44:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:44:07.571924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7324c1-4460-4651-acf1-ca8d525c6a2e service swift] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:44:07.573383 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-7e7324c1-4460-4651-acf1-ca8d525c6a2e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:44:07.586050 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7324c1-4460-4651-acf1-ca8d525c6a2e service swift] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:44:07.589669 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5def5-9e98-448e-aa7f-c662b2b97a7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:44:07.590306 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5def5-9e98-448e-aa7f-c662b2b97a7b service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:44:07.590590 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5def5-9e98-448e-aa7f-c662b2b97a7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:44:07.590865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5def5-9e98-448e-aa7f-c662b2b97a7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:44:07.592656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-c7c5def5-9e98-448e-aa7f-c662b2b97a7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:44:07.602174 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5def5-9e98-448e-aa7f-c662b2b97a7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:44:07.603626 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5def5-9e98-448e-aa7f-c662b2b97a7b service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:44:07.611865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1359/2722] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:44:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:44:08.781485 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa2005-ee67-4564-bfde-f9f31dae6485 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:44:08.782064 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa2005-ee67-4564-bfde-f9f31dae6485 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:44:08.782186 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fa2005-ee67-4564-bfde-f9f31dae6485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:44:08.800721 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-b4fa2005-ee67-4564-bfde-f9f31dae6485 None None] MFA Rules not processed for user `66ad8defb9b6453c96f1a8a7343695ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:44:08.801918 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-b4fa2005-ee67-4564-bfde-f9f31dae6485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:44:08.809033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1364/2723] 10.222.0.47 () {58 vars in 1009 bytes} [Thu Jun 30 10:44:08 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:44:08.820711 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5def5-9e98-448e-aa7f-c662b2b97a7b service neutron] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:44:08.833511 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5def5-9e98-448e-aa7f-c662b2b97a7b service neutron] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:44:08.838403 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6612a-4239-4c7b-8187-b791ca3b6bb6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07de926cb8594b098f79111dab8a9cc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:44:08.839071 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6612a-4239-4c7b-8187-b791ca3b6bb6 service swift] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:44:08.839391 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6612a-4239-4c7b-8187-b791ca3b6bb6 service swift] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:44:08.839674 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc6612a-4239-4c7b-8187-b791ca3b6bb6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:44:08.848193 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc6612a-4239-4c7b-8187-b791ca3b6bb6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:44:08.849799 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc6612a-4239-4c7b-8187-b791ca3b6bb6 service swift] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:44:08.857916 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1360/2724] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jun 30 10:44:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 30 10:44:18.795424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=35 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:44:18.854704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=12 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:44:22.170333 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5d772-356f-4348-8efe-c7da6fe0a9c6 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:44:22.170821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5d772-356f-4348-8efe-c7da6fe0a9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:44:22.170930 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa5d772-356f-4348-8efe-c7da6fe0a9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:44:22.189728 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-5fa5d772-356f-4348-8efe-c7da6fe0a9c6 None None] MFA Rules not processed for user `66ad8defb9b6453c96f1a8a7343695ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:44:22.190926 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-5fa5d772-356f-4348-8efe-c7da6fe0a9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:44:22.198089 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1365/2725] 10.222.0.47 () {58 vars in 967 bytes} [Thu Jun 30 10:44:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:44:22.210540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6612a-4239-4c7b-8187-b791ca3b6bb6 service swift] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:44:22.227643 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc6612a-4239-4c7b-8187-b791ca3b6bb6 service swift] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:44:22.233095 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44269b1b-b597-476b-bb07-2deec2ac26a7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07de926cb8594b098f79111dab8a9cc6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:44:22.233757 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44269b1b-b597-476b-bb07-2deec2ac26a7 service swift] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:44:22.234042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44269b1b-b597-476b-bb07-2deec2ac26a7 service swift] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:44:22.234393 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44269b1b-b597-476b-bb07-2deec2ac26a7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:44:22.245024 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44269b1b-b597-476b-bb07-2deec2ac26a7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:44:22.246613 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44269b1b-b597-476b-bb07-2deec2ac26a7 service swift] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:44:22.256815 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1361/2726] 10.222.0.47 () {60 vars in 1384 bytes} [Thu Jun 30 10:44:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 30 10:44:32.184244 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:44:32.253286 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=6 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:44:47.550257 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c754546-6ca0-4617-b34c-b90254ee21f8 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:44:47.550799 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c754546-6ca0-4617-b34c-b90254ee21f8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:44:47.550799 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c754546-6ca0-4617-b34c-b90254ee21f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:44:47.568497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-7c754546-6ca0-4617-b34c-b90254ee21f8 None None] MFA Rules not processed for user `fa58de3ae9eb4dc9bd80fc41ec3e0793`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:44:47.569695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7c754546-6ca0-4617-b34c-b90254ee21f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:44:47.624904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1366/2727] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:44:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:44:50.381705 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44269b1b-b597-476b-bb07-2deec2ac26a7 service swift] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:44:50.390996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44269b1b-b597-476b-bb07-2deec2ac26a7 service swift] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:44:50.395681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0141a1-535a-42f7-9a42-9130a7159421 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:44:50.396328 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0141a1-535a-42f7-9a42-9130a7159421 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:44:50.396612 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0141a1-535a-42f7-9a42-9130a7159421 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:44:50.396888 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0141a1-535a-42f7-9a42-9130a7159421 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:44:50.398651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-ce0141a1-535a-42f7-9a42-9130a7159421 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:44:50.409856 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0141a1-535a-42f7-9a42-9130a7159421 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:44:50.411286 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0141a1-535a-42f7-9a42-9130a7159421 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:44:50.420501 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1362/2728] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:44:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:44:52.385270 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c754546-6ca0-4617-b34c-b90254ee21f8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:44:52.386978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-7c754546-6ca0-4617-b34c-b90254ee21f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:44:52.406197 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c754546-6ca0-4617-b34c-b90254ee21f8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:44:52.412305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f632999-cb54-4179-8a26-4672c07f0e2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:44:52.414543 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f632999-cb54-4179-8a26-4672c07f0e2f service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:44:52.414543 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f632999-cb54-4179-8a26-4672c07f0e2f service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:44:52.414543 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f632999-cb54-4179-8a26-4672c07f0e2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:44:52.415681 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-6f632999-cb54-4179-8a26-4672c07f0e2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:44:52.429595 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f632999-cb54-4179-8a26-4672c07f0e2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:44:52.431345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f632999-cb54-4179-8a26-4672c07f0e2f service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:44:52.443663 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1367/2729] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:44:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:44:56.300315 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0141a1-535a-42f7-9a42-9130a7159421 service neutron] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:44:56.301702 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-ce0141a1-535a-42f7-9a42-9130a7159421 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:44:56.358063 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0141a1-535a-42f7-9a42-9130a7159421 service neutron] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:44:56.362704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b243fe2-137c-4029-bab3-fe759357974d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:44:56.363373 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b243fe2-137c-4029-bab3-fe759357974d service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:44:56.363721 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b243fe2-137c-4029-bab3-fe759357974d service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:44:56.364000 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b243fe2-137c-4029-bab3-fe759357974d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:44:56.365730 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-9b243fe2-137c-4029-bab3-fe759357974d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:44:56.425798 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b243fe2-137c-4029-bab3-fe759357974d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:44:56.427166 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b243fe2-137c-4029-bab3-fe759357974d service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:44:56.436871 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1363/2730] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:44:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:45:02.438133 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=31 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:45:02.867408 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f632999-cb54-4179-8a26-4672c07f0e2f service nova] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:02.880880 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f632999-cb54-4179-8a26-4672c07f0e2f service nova] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:02.887237 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad529aea-7dc4-463e-8035-bd0674610aea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:02.888043 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad529aea-7dc4-463e-8035-bd0674610aea service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:02.895948 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad529aea-7dc4-463e-8035-bd0674610aea service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:02.896278 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad529aea-7dc4-463e-8035-bd0674610aea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:02.898473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ad529aea-7dc4-463e-8035-bd0674610aea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:02.912070 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad529aea-7dc4-463e-8035-bd0674610aea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:02.913668 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad529aea-7dc4-463e-8035-bd0674610aea service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:02.924658 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1368/2731] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:45:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:45:05.405596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b243fe2-137c-4029-bab3-fe759357974d service cinder] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:05.415640 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b243fe2-137c-4029-bab3-fe759357974d service cinder] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:05.419548 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6ea84-f414-4c92-b299-330b253f561a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:05.424772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6ea84-f414-4c92-b299-330b253f561a service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:05.424772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6ea84-f414-4c92-b299-330b253f561a service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:05.424772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6ea84-f414-4c92-b299-330b253f561a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:05.424772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-b0b6ea84-f414-4c92-b299-330b253f561a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:05.489654 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b6ea84-f414-4c92-b299-330b253f561a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:05.491489 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b6ea84-f414-4c92-b299-330b253f561a service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:05.502102 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1364/2732] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:45:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:45:08.782093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad529aea-7dc4-463e-8035-bd0674610aea service nova] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:08.795141 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad529aea-7dc4-463e-8035-bd0674610aea service nova] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:08.803986 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d597f-7651-405f-9190-892af6b7e548 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:08.803986 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d597f-7651-405f-9190-892af6b7e548 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:08.803986 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d597f-7651-405f-9190-892af6b7e548 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:08.803986 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0d597f-7651-405f-9190-892af6b7e548 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:08.805506 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fb0d597f-7651-405f-9190-892af6b7e548 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:08.819384 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d597f-7651-405f-9190-892af6b7e548 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:08.821275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0d597f-7651-405f-9190-892af6b7e548 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:08.833390 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1369/2733] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:45:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:45:11.062355 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6ea84-f414-4c92-b299-330b253f561a service neutron] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:11.063924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-b0b6ea84-f414-4c92-b299-330b253f561a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:11.076600 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6ea84-f414-4c92-b299-330b253f561a service neutron] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:11.082244 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c48fd-4873-4d4d-87a8-8d51a924e2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:11.083060 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c48fd-4873-4d4d-87a8-8d51a924e2f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:11.083374 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c48fd-4873-4d4d-87a8-8d51a924e2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:11.083656 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c48fd-4873-4d4d-87a8-8d51a924e2f3 admin admin] PATH_INFO: `/v3/users/1f4a95c82152441096c50f79acc68e9a` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:11.084861 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c48fd-4873-4d4d-87a8-8d51a924e2f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f4a95c82152441096c50f79acc68e9a)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:11.086843 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c48fd-4873-4d4d-87a8-8d51a924e2f3 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:11.380666 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:45:11.393751 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bb0c48fd-4873-4d4d-87a8-8d51a924e2f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f4a95c82152441096c50f79acc68e9a'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:11.582002 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bb0c48fd-4873-4d4d-87a8-8d51a924e2f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f4a95c82152441096c50f79acc68e9a'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:11.595054 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bb0c48fd-4873-4d4d-87a8-8d51a924e2f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f4a95c82152441096c50f79acc68e9a'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:11.595778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bb0c48fd-4873-4d4d-87a8-8d51a924e2f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f4a95c82152441096c50f79acc68e9a'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:11.596447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bb0c48fd-4873-4d4d-87a8-8d51a924e2f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f4a95c82152441096c50f79acc68e9a'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:11.609482 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1365/2734] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:45:11 2022] DELETE /identity/v3/users/1f4a95c82152441096c50f79acc68e9a => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:45:11.906322 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d597f-7651-405f-9190-892af6b7e548 service nova] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:11.918695 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0d597f-7651-405f-9190-892af6b7e548 service nova] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:11.978105 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91eb585-3243-4804-85f8-49dfedc781fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:11.978968 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91eb585-3243-4804-85f8-49dfedc781fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:11.979254 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91eb585-3243-4804-85f8-49dfedc781fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:11.979555 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91eb585-3243-4804-85f8-49dfedc781fd admin admin] PATH_INFO: `/v3/projects/3b23261b1f1944d4a3bbe630bcde4d61` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:11.980724 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91eb585-3243-4804-85f8-49dfedc781fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b23261b1f1944d4a3bbe630bcde4d61)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:11.982926 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91eb585-3243-4804-85f8-49dfedc781fd admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:12.130368 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b91eb585-3243-4804-85f8-49dfedc781fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b23261b1f1944d4a3bbe630bcde4d61', 'request_id': 'req-b91eb585-3243-4804-85f8-49dfedc781fd'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:12.225188 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b91eb585-3243-4804-85f8-49dfedc781fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b23261b1f1944d4a3bbe630bcde4d61', 'request_id': 'req-b91eb585-3243-4804-85f8-49dfedc781fd'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:12.231529 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b91eb585-3243-4804-85f8-49dfedc781fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b23261b1f1944d4a3bbe630bcde4d61', 'request_id': 'req-b91eb585-3243-4804-85f8-49dfedc781fd'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:12.242797 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b91eb585-3243-4804-85f8-49dfedc781fd admin admin] The token cache is being invalidate because project 3b23261b1f1944d4a3bbe630bcde4d61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:45:12.243089 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b91eb585-3243-4804-85f8-49dfedc781fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:12.243754 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-b91eb585-3243-4804-85f8-49dfedc781fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:12.245304 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1370/2735] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:45:11 2022] DELETE /identity/v3/projects/3b23261b1f1944d4a3bbe630bcde4d61 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:45:12.250306 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd2334-7462-4983-afe4-49abbe700519 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:12.250824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd2334-7462-4983-afe4-49abbe700519 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:12.250926 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd2334-7462-4983-afe4-49abbe700519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:12.269521 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-28cd2334-7462-4983-afe4-49abbe700519 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:45:12.270890 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-28cd2334-7462-4983-afe4-49abbe700519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:12.330602 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1366/2736] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:45:12.338345 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e0330-ff4d-4704-8eb5-eb4510900334 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:12.338645 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e0330-ff4d-4704-8eb5-eb4510900334 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:12.338931 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e0330-ff4d-4704-8eb5-eb4510900334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:12.356868 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-043e0330-ff4d-4704-8eb5-eb4510900334 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:45:12.358086 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-043e0330-ff4d-4704-8eb5-eb4510900334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:12.366834 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1371/2737] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:45:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:45:12.370947 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd2334-7462-4983-afe4-49abbe700519 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:12.381352 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd2334-7462-4983-afe4-49abbe700519 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:12.385951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff55213-96b0-4ee9-80e1-3b84bf24a201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:12.386611 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff55213-96b0-4ee9-80e1-3b84bf24a201 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:12.386894 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff55213-96b0-4ee9-80e1-3b84bf24a201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:12.387173 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff55213-96b0-4ee9-80e1-3b84bf24a201 admin admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:12.387931 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff55213-96b0-4ee9-80e1-3b84bf24a201 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:45:12.388242 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff55213-96b0-4ee9-80e1-3b84bf24a201 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:12.389841 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff55213-96b0-4ee9-80e1-3b84bf24a201 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:12.401540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1367/2738] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:45:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:45:12.405753 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e0330-ff4d-4704-8eb5-eb4510900334 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:12.415558 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e0330-ff4d-4704-8eb5-eb4510900334 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:12.420203 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4314f-06c7-450f-b0a4-f2e29d1129e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:12.420867 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4314f-06c7-450f-b0a4-f2e29d1129e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:12.421156 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4314f-06c7-450f-b0a4-f2e29d1129e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:12.421438 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4314f-06c7-450f-b0a4-f2e29d1129e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:12.422144 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4314f-06c7-450f-b0a4-f2e29d1129e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:12.424122 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f4314f-06c7-450f-b0a4-f2e29d1129e8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:12.589078 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1372/2739] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:45:12 2022] POST /identity/v3/projects => generated 371 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:45:12.593126 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff55213-96b0-4ee9-80e1-3b84bf24a201 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:12.603065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff55213-96b0-4ee9-80e1-3b84bf24a201 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:12.661961 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8784b4-9eff-415b-a4a5-6cd167f6faee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:12.662645 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8784b4-9eff-415b-a4a5-6cd167f6faee admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:12.662946 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8784b4-9eff-415b-a4a5-6cd167f6faee admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:12.663214 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8784b4-9eff-415b-a4a5-6cd167f6faee admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:12.664200 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8784b4-9eff-415b-a4a5-6cd167f6faee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:12.666242 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8784b4-9eff-415b-a4a5-6cd167f6faee admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:12.820001 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1368/2740] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:45:12 2022] POST /identity/v3/users => generated 286 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:45:12.823994 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4314f-06c7-450f-b0a4-f2e29d1129e8 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:12.834356 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f4314f-06c7-450f-b0a4-f2e29d1129e8 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:12.838818 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e22ac6-eeba-47b5-b18f-6105c16b9a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:12.839516 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e22ac6-eeba-47b5-b18f-6105c16b9a8b admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:12.839802 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e22ac6-eeba-47b5-b18f-6105c16b9a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:12.840080 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e22ac6-eeba-47b5-b18f-6105c16b9a8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:12.840811 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e22ac6-eeba-47b5-b18f-6105c16b9a8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:12.842448 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e22ac6-eeba-47b5-b18f-6105c16b9a8b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:12.853566 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1373/2741] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:45:12.864189 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8784b4-9eff-415b-a4a5-6cd167f6faee admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:12.874210 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8784b4-9eff-415b-a4a5-6cd167f6faee admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:12.879127 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c8d71-c699-4b5f-b7e4-a0a87fc786b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:12.879836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c8d71-c699-4b5f-b7e4-a0a87fc786b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:12.880122 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c8d71-c699-4b5f-b7e4-a0a87fc786b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:12.880400 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c8d71-c699-4b5f-b7e4-a0a87fc786b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:12.881207 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2c8d71-c699-4b5f-b7e4-a0a87fc786b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:12.882831 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2c8d71-c699-4b5f-b7e4-a0a87fc786b8 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:12.893151 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1369/2742] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:45:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:45:12.897221 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e22ac6-eeba-47b5-b18f-6105c16b9a8b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:12.907565 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e22ac6-eeba-47b5-b18f-6105c16b9a8b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:12.912781 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dac433-61e3-44d8-8dc3-a532621a844c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:12.913667 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dac433-61e3-44d8-8dc3-a532621a844c admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:12.914210 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dac433-61e3-44d8-8dc3-a532621a844c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:12.914484 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dac433-61e3-44d8-8dc3-a532621a844c admin admin] PATH_INFO: `/v3/projects/0bff9dae08634c3791e3c85837416ae9/users/c01bf0c1085548cca31099dbf48f6c42/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:12.929763 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dac433-61e3-44d8-8dc3-a532621a844c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bff9dae08634c3791e3c85837416ae9, user_id=c01bf0c1085548cca31099dbf48f6c42, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:12.932467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dac433-61e3-44d8-8dc3-a532621a844c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:13.062793 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:45:13.067934 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1374/2743] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:45:12 2022] PUT /identity/v3/projects/0bff9dae08634c3791e3c85837416ae9/users/c01bf0c1085548cca31099dbf48f6c42/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:45:13.080324 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d7bb4-ad83-4b3c-8ba5-090a319c3c69 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:13.080324 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d7bb4-ad83-4b3c-8ba5-090a319c3c69 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:13.080324 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d7bb4-ad83-4b3c-8ba5-090a319c3c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:13.103759 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:45:13.106829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-658d7bb4-ad83-4b3c-8ba5-090a319c3c69 None None] MFA Rules not processed for user `c01bf0c1085548cca31099dbf48f6c42`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:45:13.108112 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-658d7bb4-ad83-4b3c-8ba5-090a319c3c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:13.171899 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1370/2744] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:45:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:45:13.183705 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dac433-61e3-44d8-8dc3-a532621a844c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:13.185370 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-75dac433-61e3-44d8-8dc3-a532621a844c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:13.258245 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dac433-61e3-44d8-8dc3-a532621a844c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:13.263887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da130a1-9819-4336-b794-dd55674d8960 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71f6d7715e6f49ffb9fdf6e1307bbd51', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:13.264535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da130a1-9819-4336-b794-dd55674d8960 service nova] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:13.264821 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da130a1-9819-4336-b794-dd55674d8960 service nova] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:13.265097 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da130a1-9819-4336-b794-dd55674d8960 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:13.273577 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da130a1-9819-4336-b794-dd55674d8960 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:13.275187 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da130a1-9819-4336-b794-dd55674d8960 service nova] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:13.284007 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1375/2745] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:45:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:45:16.911016 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d7bb4-ad83-4b3c-8ba5-090a319c3c69 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:16.969945 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d7bb4-ad83-4b3c-8ba5-090a319c3c69 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:16.974550 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa74e875-e903-4e5d-9d5b-7c83eb011713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:16.975454 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74e875-e903-4e5d-9d5b-7c83eb011713 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:16.975742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74e875-e903-4e5d-9d5b-7c83eb011713 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:16.976046 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa74e875-e903-4e5d-9d5b-7c83eb011713 admin admin] PATH_INFO: `/v3/users/c01bf0c1085548cca31099dbf48f6c42` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:16.977279 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74e875-e903-4e5d-9d5b-7c83eb011713 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c01bf0c1085548cca31099dbf48f6c42)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:16.979231 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa74e875-e903-4e5d-9d5b-7c83eb011713 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:17.096713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-aa74e875-e903-4e5d-9d5b-7c83eb011713 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c01bf0c1085548cca31099dbf48f6c42'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:17.147065 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-aa74e875-e903-4e5d-9d5b-7c83eb011713 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c01bf0c1085548cca31099dbf48f6c42'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:17.158890 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-aa74e875-e903-4e5d-9d5b-7c83eb011713 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c01bf0c1085548cca31099dbf48f6c42'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:17.160201 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-aa74e875-e903-4e5d-9d5b-7c83eb011713 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c01bf0c1085548cca31099dbf48f6c42'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:17.160732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-aa74e875-e903-4e5d-9d5b-7c83eb011713 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c01bf0c1085548cca31099dbf48f6c42'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:17.173013 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1371/2746] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:45:16 2022] DELETE /identity/v3/users/c01bf0c1085548cca31099dbf48f6c42 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:45:17.178352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f597ef-7335-49a6-bba7-c26257cdafe8 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:17.178868 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f597ef-7335-49a6-bba7-c26257cdafe8 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:17.178978 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f597ef-7335-49a6-bba7-c26257cdafe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:17.196180 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-66f597ef-7335-49a6-bba7-c26257cdafe8 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:45:17.197373 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-66f597ef-7335-49a6-bba7-c26257cdafe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:17.256807 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1376/2747] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:45:17.266585 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa74e875-e903-4e5d-9d5b-7c83eb011713 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:17.268021 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-aa74e875-e903-4e5d-9d5b-7c83eb011713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:17.295080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:45:17.325468 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa74e875-e903-4e5d-9d5b-7c83eb011713 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:17.329132 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd42ec39-6595-47e1-9c97-331fdd5dac17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:17.329780 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42ec39-6595-47e1-9c97-331fdd5dac17 service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:17.330071 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42ec39-6595-47e1-9c97-331fdd5dac17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:17.330355 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd42ec39-6595-47e1-9c97-331fdd5dac17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:17.339414 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42ec39-6595-47e1-9c97-331fdd5dac17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:17.344448 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd42ec39-6595-47e1-9c97-331fdd5dac17 service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:17.350802 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1372/2748] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:45:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:45:17.789729 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f597ef-7335-49a6-bba7-c26257cdafe8 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:17.796002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-66f597ef-7335-49a6-bba7-c26257cdafe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:17.809233 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f597ef-7335-49a6-bba7-c26257cdafe8 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:17.813931 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d6e30c-e60b-4ea1-9f6f-5afff5460851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:17.814870 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6e30c-e60b-4ea1-9f6f-5afff5460851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:17.815158 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6e30c-e60b-4ea1-9f6f-5afff5460851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:17.818890 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d6e30c-e60b-4ea1-9f6f-5afff5460851 admin admin] PATH_INFO: `/v3/projects/0bff9dae08634c3791e3c85837416ae9` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:17.818890 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d6e30c-e60b-4ea1-9f6f-5afff5460851 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bff9dae08634c3791e3c85837416ae9)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:17.818890 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d6e30c-e60b-4ea1-9f6f-5afff5460851 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:17.941216 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-61d6e30c-e60b-4ea1-9f6f-5afff5460851 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bff9dae08634c3791e3c85837416ae9', 'request_id': 'req-61d6e30c-e60b-4ea1-9f6f-5afff5460851'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:17.998304 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-61d6e30c-e60b-4ea1-9f6f-5afff5460851 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bff9dae08634c3791e3c85837416ae9', 'request_id': 'req-61d6e30c-e60b-4ea1-9f6f-5afff5460851'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:18.005125 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-61d6e30c-e60b-4ea1-9f6f-5afff5460851 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bff9dae08634c3791e3c85837416ae9', 'request_id': 'req-61d6e30c-e60b-4ea1-9f6f-5afff5460851'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:18.016274 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-61d6e30c-e60b-4ea1-9f6f-5afff5460851 admin admin] The token cache is being invalidate because project 0bff9dae08634c3791e3c85837416ae9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:45:18.016567 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-61d6e30c-e60b-4ea1-9f6f-5afff5460851 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:18.017198 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-61d6e30c-e60b-4ea1-9f6f-5afff5460851 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:18.019346 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1377/2749] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:45:17 2022] DELETE /identity/v3/projects/0bff9dae08634c3791e3c85837416ae9 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:45:18.024352 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cace25-4a87-494a-84af-6f02ae4d3009 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:18.024837 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cace25-4a87-494a-84af-6f02ae4d3009 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:18.024944 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cace25-4a87-494a-84af-6f02ae4d3009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:18.042681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-82cace25-4a87-494a-84af-6f02ae4d3009 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:45:18.043855 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-82cace25-4a87-494a-84af-6f02ae4d3009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:18.104864 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1373/2750] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:45:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:45:18.113520 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e64b8-b22e-4751-93a6-bb223d3d3eea None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:18.113802 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e64b8-b22e-4751-93a6-bb223d3d3eea None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:18.119101 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320e64b8-b22e-4751-93a6-bb223d3d3eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:18.132901 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-320e64b8-b22e-4751-93a6-bb223d3d3eea None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:45:18.134098 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-320e64b8-b22e-4751-93a6-bb223d3d3eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:18.141986 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1378/2751] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:45:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:45:18.147754 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cace25-4a87-494a-84af-6f02ae4d3009 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:18.156367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cace25-4a87-494a-84af-6f02ae4d3009 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:18.160983 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8001c97-dbb0-4a35-b883-74c493cdf95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:18.161631 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8001c97-dbb0-4a35-b883-74c493cdf95d admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:18.161913 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8001c97-dbb0-4a35-b883-74c493cdf95d admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:18.162189 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8001c97-dbb0-4a35-b883-74c493cdf95d admin admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:18.162943 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8001c97-dbb0-4a35-b883-74c493cdf95d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:45:18.163250 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8001c97-dbb0-4a35-b883-74c493cdf95d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:18.165122 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8001c97-dbb0-4a35-b883-74c493cdf95d admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:18.176462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1374/2752] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:45:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:45:18.180681 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e64b8-b22e-4751-93a6-bb223d3d3eea None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:18.187911 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:45:18.191030 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320e64b8-b22e-4751-93a6-bb223d3d3eea None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:18.195887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47290a5-214c-4837-b9b6-3b00a4c55098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:18.196537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47290a5-214c-4837-b9b6-3b00a4c55098 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:18.196824 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47290a5-214c-4837-b9b6-3b00a4c55098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:18.197104 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47290a5-214c-4837-b9b6-3b00a4c55098 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:18.197806 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47290a5-214c-4837-b9b6-3b00a4c55098 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:18.199768 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47290a5-214c-4837-b9b6-3b00a4c55098 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:18.279993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1379/2753] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:45:18 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:45:18.284766 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8001c97-dbb0-4a35-b883-74c493cdf95d admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:18.294945 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8001c97-dbb0-4a35-b883-74c493cdf95d admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:18.351099 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4746ff7-050f-4042-a28e-b4f0086157c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:18.351797 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4746ff7-050f-4042-a28e-b4f0086157c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:18.352081 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4746ff7-050f-4042-a28e-b4f0086157c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:18.352355 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4746ff7-050f-4042-a28e-b4f0086157c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:18.353042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4746ff7-050f-4042-a28e-b4f0086157c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:18.354949 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4746ff7-050f-4042-a28e-b4f0086157c5 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:18.464979 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1375/2754] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:45:18 2022] POST /identity/v3/users => generated 290 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:45:18.469083 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47290a5-214c-4837-b9b6-3b00a4c55098 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:18.479750 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47290a5-214c-4837-b9b6-3b00a4c55098 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:18.484455 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9a46b-7312-493f-85b0-fda0615d0a62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:18.485119 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9a46b-7312-493f-85b0-fda0615d0a62 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:18.485406 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9a46b-7312-493f-85b0-fda0615d0a62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:18.485688 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9a46b-7312-493f-85b0-fda0615d0a62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:18.486476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9a46b-7312-493f-85b0-fda0615d0a62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:18.488645 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af9a46b-7312-493f-85b0-fda0615d0a62 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:18.501196 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1380/2755] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:45:18.502820 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4746ff7-050f-4042-a28e-b4f0086157c5 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:18.516038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4746ff7-050f-4042-a28e-b4f0086157c5 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:18.521099 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309d8953-f5d7-498c-94fe-208251d6d9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:18.521771 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d8953-f5d7-498c-94fe-208251d6d9f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:18.522060 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d8953-f5d7-498c-94fe-208251d6d9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:18.522345 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309d8953-f5d7-498c-94fe-208251d6d9f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:18.523079 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d8953-f5d7-498c-94fe-208251d6d9f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:18.524724 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309d8953-f5d7-498c-94fe-208251d6d9f3 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:18.535371 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1376/2756] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 30 10:45:18.539792 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9a46b-7312-493f-85b0-fda0615d0a62 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:18.550880 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9a46b-7312-493f-85b0-fda0615d0a62 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:18.555073 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc53404-5f22-4928-b8a2-cbf24b1a78d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:18.556007 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc53404-5f22-4928-b8a2-cbf24b1a78d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:18.556286 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc53404-5f22-4928-b8a2-cbf24b1a78d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:18.556561 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc53404-5f22-4928-b8a2-cbf24b1a78d3 admin admin] PATH_INFO: `/v3/projects/35ae7bc294d4413bb3511994ebb81350/users/c8e4232bbd884c448fc24d6b86d49a36/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:18.572497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc53404-5f22-4928-b8a2-cbf24b1a78d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35ae7bc294d4413bb3511994ebb81350, user_id=c8e4232bbd884c448fc24d6b86d49a36, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:18.575141 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc53404-5f22-4928-b8a2-cbf24b1a78d3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:18.684027 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1381/2757] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:45:18 2022] PUT /identity/v3/projects/35ae7bc294d4413bb3511994ebb81350/users/c8e4232bbd884c448fc24d6b86d49a36/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:45:18.694297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3c007-28e5-4ac7-a180-32d6858664a3 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:18.694617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3c007-28e5-4ac7-a180-32d6858664a3 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:18.694899 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3c007-28e5-4ac7-a180-32d6858664a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:18.721173 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-edd3c007-28e5-4ac7-a180-32d6858664a3 None None] MFA Rules not processed for user `c8e4232bbd884c448fc24d6b86d49a36`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:45:18.723007 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-edd3c007-28e5-4ac7-a180-32d6858664a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:18.741678 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:45:18.787632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1377/2758] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:45:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:45:18.791955 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc53404-5f22-4928-b8a2-cbf24b1a78d3 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:18.855798 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc53404-5f22-4928-b8a2-cbf24b1a78d3 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:18.860471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8623acf9-e50a-4362-83d8-dc4e38615aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:18.861139 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623acf9-e50a-4362-83d8-dc4e38615aea admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:18.861428 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623acf9-e50a-4362-83d8-dc4e38615aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:18.861712 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8623acf9-e50a-4362-83d8-dc4e38615aea admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:18.863034 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8623acf9-e50a-4362-83d8-dc4e38615aea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:18.865067 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8623acf9-e50a-4362-83d8-dc4e38615aea admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:18.970880 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1382/2759] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:45:18 2022] POST /identity/v3/projects => generated 381 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:45:18.975570 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3c007-28e5-4ac7-a180-32d6858664a3 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:18.986369 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3c007-28e5-4ac7-a180-32d6858664a3 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:19.042307 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199f149-65dc-4069-ab78-3c44c9e291d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:19.042973 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199f149-65dc-4069-ab78-3c44c9e291d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:19.043249 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199f149-65dc-4069-ab78-3c44c9e291d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:19.043554 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9199f149-65dc-4069-ab78-3c44c9e291d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:19.044242 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9199f149-65dc-4069-ab78-3c44c9e291d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:19.046191 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9199f149-65dc-4069-ab78-3c44c9e291d4 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:19.118352 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1378/2760] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:45:18 2022] POST /identity/v3/users => generated 297 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:45:19.122230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8623acf9-e50a-4362-83d8-dc4e38615aea admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:19.132772 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8623acf9-e50a-4362-83d8-dc4e38615aea admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:19.137366 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf5378-4509-4b23-802b-46bcf25df039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:19.138028 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf5378-4509-4b23-802b-46bcf25df039 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:19.138311 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf5378-4509-4b23-802b-46bcf25df039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:19.138589 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcf5378-4509-4b23-802b-46bcf25df039 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:19.139342 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf5378-4509-4b23-802b-46bcf25df039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:19.140944 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcf5378-4509-4b23-802b-46bcf25df039 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:19.151945 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1383/2761] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:45:19.155956 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199f149-65dc-4069-ab78-3c44c9e291d4 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:19.166617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9199f149-65dc-4069-ab78-3c44c9e291d4 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:19.171179 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb908d-2965-4503-9d67-5ccb7b23240e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:19.172088 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb908d-2965-4503-9d67-5ccb7b23240e admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:19.172373 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb908d-2965-4503-9d67-5ccb7b23240e admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:19.172651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb908d-2965-4503-9d67-5ccb7b23240e admin admin] PATH_INFO: `/v3/projects/188ff1e31e6e478dbda3e3f95c6eb306/users/7353317fa2a54d82a58b8e7077f7bb89/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:19.187367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb908d-2965-4503-9d67-5ccb7b23240e admin admin] RBAC: Authorizing `identity:create_grant(project_id=188ff1e31e6e478dbda3e3f95c6eb306, user_id=7353317fa2a54d82a58b8e7077f7bb89, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:19.190024 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb908d-2965-4503-9d67-5ccb7b23240e admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:19.427902 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1379/2762] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:45:19 2022] PUT /identity/v3/projects/188ff1e31e6e478dbda3e3f95c6eb306/users/7353317fa2a54d82a58b8e7077f7bb89/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:45:19.431986 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf5378-4509-4b23-802b-46bcf25df039 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:19.442079 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcf5378-4509-4b23-802b-46bcf25df039 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:19.498716 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6611e53f-4de5-4047-9e52-6ab68495ef17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:19.499429 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611e53f-4de5-4047-9e52-6ab68495ef17 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:19.499735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611e53f-4de5-4047-9e52-6ab68495ef17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:19.500040 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611e53f-4de5-4047-9e52-6ab68495ef17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:19.500778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6611e53f-4de5-4047-9e52-6ab68495ef17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:19.502392 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6611e53f-4de5-4047-9e52-6ab68495ef17 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:19.512667 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1384/2763] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:45:19.516948 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb908d-2965-4503-9d67-5ccb7b23240e admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:19.527097 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb908d-2965-4503-9d67-5ccb7b23240e admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:19.531972 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f10f5-4458-48d5-a409-31d49c597952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:19.532854 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f10f5-4458-48d5-a409-31d49c597952 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:19.533143 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f10f5-4458-48d5-a409-31d49c597952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:19.533424 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07f10f5-4458-48d5-a409-31d49c597952 admin admin] PATH_INFO: `/v3/projects/188ff1e31e6e478dbda3e3f95c6eb306/users/7353317fa2a54d82a58b8e7077f7bb89/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:19.536454 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07f10f5-4458-48d5-a409-31d49c597952 admin admin] RBAC: Authorizing `identity:create_grant(project_id=188ff1e31e6e478dbda3e3f95c6eb306, user_id=7353317fa2a54d82a58b8e7077f7bb89, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:19.539387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07f10f5-4458-48d5-a409-31d49c597952 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:19.549741 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1380/2764] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:45:19 2022] PUT /identity/v3/projects/188ff1e31e6e478dbda3e3f95c6eb306/users/7353317fa2a54d82a58b8e7077f7bb89/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:45:19.566649 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f9124-18b7-41e9-bd7a-20490e7958f3 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:19.566942 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f9124-18b7-41e9-bd7a-20490e7958f3 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:19.567222 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963f9124-18b7-41e9-bd7a-20490e7958f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:19.592739 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-963f9124-18b7-41e9-bd7a-20490e7958f3 None None] MFA Rules not processed for user `7353317fa2a54d82a58b8e7077f7bb89`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:45:19.593934 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-963f9124-18b7-41e9-bd7a-20490e7958f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:19.598485 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:45:19.663242 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1385/2765] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:45:19 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:45:19.676865 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f10f5-4458-48d5-a409-31d49c597952 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:19.678540 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d07f10f5-4458-48d5-a409-31d49c597952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:19.735196 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07f10f5-4458-48d5-a409-31d49c597952 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:19.739546 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b73a7-a48d-4edf-a6bb-f21a95c69a39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:19.740197 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b73a7-a48d-4edf-a6bb-f21a95c69a39 service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:19.740483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b73a7-a48d-4edf-a6bb-f21a95c69a39 service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:19.740782 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b73a7-a48d-4edf-a6bb-f21a95c69a39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:19.749707 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b73a7-a48d-4edf-a6bb-f21a95c69a39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:19.751144 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b73a7-a48d-4edf-a6bb-f21a95c69a39 service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:19.760624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1381/2766] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:45:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:45:20.005166 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f9124-18b7-41e9-bd7a-20490e7958f3 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:20.014780 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963f9124-18b7-41e9-bd7a-20490e7958f3 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:20.018893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80389c79-84b2-4097-b858-e68f34ea6c4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:20.019579 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80389c79-84b2-4097-b858-e68f34ea6c4c service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:20.019863 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80389c79-84b2-4097-b858-e68f34ea6c4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:20.020138 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80389c79-84b2-4097-b858-e68f34ea6c4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:20.075087 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80389c79-84b2-4097-b858-e68f34ea6c4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:20.076570 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80389c79-84b2-4097-b858-e68f34ea6c4c service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:20.086008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1386/2767] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:45:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:45:29.756676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=88,keystone:INSERT=5 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:45:30.081395 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=55 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:45:33.657194 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b73a7-a48d-4edf-a6bb-f21a95c69a39 service cinder] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:33.721791 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b73a7-a48d-4edf-a6bb-f21a95c69a39 service cinder] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:33.727668 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea37ce-090c-4884-aa35-c7917248db06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:33.728603 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea37ce-090c-4884-aa35-c7917248db06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:33.728888 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea37ce-090c-4884-aa35-c7917248db06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:33.729167 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ea37ce-090c-4884-aa35-c7917248db06 admin admin] PATH_INFO: `/v3/users/c8e4232bbd884c448fc24d6b86d49a36` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:33.730366 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea37ce-090c-4884-aa35-c7917248db06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8e4232bbd884c448fc24d6b86d49a36)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:33.732350 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ea37ce-090c-4884-aa35-c7917248db06 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:33.943450 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-97ea37ce-090c-4884-aa35-c7917248db06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8e4232bbd884c448fc24d6b86d49a36'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:34.067364 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-97ea37ce-090c-4884-aa35-c7917248db06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8e4232bbd884c448fc24d6b86d49a36'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:34.079783 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-97ea37ce-090c-4884-aa35-c7917248db06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8e4232bbd884c448fc24d6b86d49a36'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:34.080483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-97ea37ce-090c-4884-aa35-c7917248db06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8e4232bbd884c448fc24d6b86d49a36'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:34.081089 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-97ea37ce-090c-4884-aa35-c7917248db06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8e4232bbd884c448fc24d6b86d49a36'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:34.094398 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1382/2768] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:45:33 2022] DELETE /identity/v3/users/c8e4232bbd884c448fc24d6b86d49a36 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:45:34.100081 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb723ae-f12d-42ac-b13b-d4999505ef4b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:34.100609 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb723ae-f12d-42ac-b13b-d4999505ef4b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:34.100609 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb723ae-f12d-42ac-b13b-d4999505ef4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:34.119225 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-8fb723ae-f12d-42ac-b13b-d4999505ef4b None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:45:34.120473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8fb723ae-f12d-42ac-b13b-d4999505ef4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:34.183899 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1387/2769] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:45:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:45:34.193238 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea37ce-090c-4884-aa35-c7917248db06 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:34.194701 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-97ea37ce-090c-4884-aa35-c7917248db06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:34.253058 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ea37ce-090c-4884-aa35-c7917248db06 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:34.256603 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa630af4-8465-494b-a4c8-5fe639e874bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:34.257200 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa630af4-8465-494b-a4c8-5fe639e874bc service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:34.257489 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa630af4-8465-494b-a4c8-5fe639e874bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:34.257772 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa630af4-8465-494b-a4c8-5fe639e874bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:34.267040 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa630af4-8465-494b-a4c8-5fe639e874bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:34.268523 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa630af4-8465-494b-a4c8-5fe639e874bc service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:34.278895 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1383/2770] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:45:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:45:35.073497 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb723ae-f12d-42ac-b13b-d4999505ef4b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:35.075047 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8fb723ae-f12d-42ac-b13b-d4999505ef4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:35.153331 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb723ae-f12d-42ac-b13b-d4999505ef4b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:35.158039 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0357a80-0507-4aac-b06c-7ca1435aae3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:35.158918 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0357a80-0507-4aac-b06c-7ca1435aae3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:35.159200 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0357a80-0507-4aac-b06c-7ca1435aae3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:35.159508 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0357a80-0507-4aac-b06c-7ca1435aae3a admin admin] PATH_INFO: `/v3/projects/35ae7bc294d4413bb3511994ebb81350` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:35.160974 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0357a80-0507-4aac-b06c-7ca1435aae3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=35ae7bc294d4413bb3511994ebb81350)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:35.162907 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0357a80-0507-4aac-b06c-7ca1435aae3a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:35.339090 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a0357a80-0507-4aac-b06c-7ca1435aae3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35ae7bc294d4413bb3511994ebb81350', 'request_id': 'req-a0357a80-0507-4aac-b06c-7ca1435aae3a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:35.420667 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a0357a80-0507-4aac-b06c-7ca1435aae3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35ae7bc294d4413bb3511994ebb81350', 'request_id': 'req-a0357a80-0507-4aac-b06c-7ca1435aae3a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:35.426460 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a0357a80-0507-4aac-b06c-7ca1435aae3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35ae7bc294d4413bb3511994ebb81350', 'request_id': 'req-a0357a80-0507-4aac-b06c-7ca1435aae3a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:35.437533 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a0357a80-0507-4aac-b06c-7ca1435aae3a admin admin] The token cache is being invalidate because project 35ae7bc294d4413bb3511994ebb81350 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:45:35.437815 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a0357a80-0507-4aac-b06c-7ca1435aae3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:35.438465 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a0357a80-0507-4aac-b06c-7ca1435aae3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:35.440039 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1388/2771] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:45:35 2022] DELETE /identity/v3/projects/35ae7bc294d4413bb3511994ebb81350 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:45:35.444679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa630af4-8465-494b-a4c8-5fe639e874bc service neutron] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:35.469848 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:45:35.488672 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa630af4-8465-494b-a4c8-5fe639e874bc service neutron] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:35.549690 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f074c-f07f-4c41-8b4b-583fe243eafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:35.551169 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f074c-f07f-4c41-8b4b-583fe243eafc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:35.551547 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f074c-f07f-4c41-8b4b-583fe243eafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:35.551835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f074c-f07f-4c41-8b4b-583fe243eafc admin admin] PATH_INFO: `/v3/users/7353317fa2a54d82a58b8e7077f7bb89` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:35.553051 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f074c-f07f-4c41-8b4b-583fe243eafc admin admin] RBAC: Authorizing `identity:delete_user(user_id=7353317fa2a54d82a58b8e7077f7bb89)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:35.554972 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f074c-f07f-4c41-8b4b-583fe243eafc admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:35.870028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-0f3f074c-f07f-4c41-8b4b-583fe243eafc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7353317fa2a54d82a58b8e7077f7bb89'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:35.969401 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-0f3f074c-f07f-4c41-8b4b-583fe243eafc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7353317fa2a54d82a58b8e7077f7bb89'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:35.982225 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-0f3f074c-f07f-4c41-8b4b-583fe243eafc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7353317fa2a54d82a58b8e7077f7bb89'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:35.982855 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-0f3f074c-f07f-4c41-8b4b-583fe243eafc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7353317fa2a54d82a58b8e7077f7bb89'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:35.983483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-0f3f074c-f07f-4c41-8b4b-583fe243eafc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7353317fa2a54d82a58b8e7077f7bb89'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:35.996176 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1384/2772] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:45:35 2022] DELETE /identity/v3/users/7353317fa2a54d82a58b8e7077f7bb89 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:45:36.468315 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0357a80-0507-4aac-b06c-7ca1435aae3a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:36.469924 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a0357a80-0507-4aac-b06c-7ca1435aae3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:36.486278 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0357a80-0507-4aac-b06c-7ca1435aae3a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:36.507666 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:45:36.548113 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71525e8-97c8-44e7-911d-98475ea2aac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:36.548987 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71525e8-97c8-44e7-911d-98475ea2aac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:36.549271 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71525e8-97c8-44e7-911d-98475ea2aac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:36.549547 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71525e8-97c8-44e7-911d-98475ea2aac3 admin admin] PATH_INFO: `/v3/projects/188ff1e31e6e478dbda3e3f95c6eb306` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:36.550738 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71525e8-97c8-44e7-911d-98475ea2aac3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=188ff1e31e6e478dbda3e3f95c6eb306)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:36.553182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71525e8-97c8-44e7-911d-98475ea2aac3 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:36.661869 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f71525e8-97c8-44e7-911d-98475ea2aac3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '188ff1e31e6e478dbda3e3f95c6eb306', 'request_id': 'req-f71525e8-97c8-44e7-911d-98475ea2aac3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:36.734837 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f71525e8-97c8-44e7-911d-98475ea2aac3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '188ff1e31e6e478dbda3e3f95c6eb306', 'request_id': 'req-f71525e8-97c8-44e7-911d-98475ea2aac3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:36.740675 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f71525e8-97c8-44e7-911d-98475ea2aac3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '188ff1e31e6e478dbda3e3f95c6eb306', 'request_id': 'req-f71525e8-97c8-44e7-911d-98475ea2aac3'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:36.751646 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f71525e8-97c8-44e7-911d-98475ea2aac3 admin admin] The token cache is being invalidate because project 188ff1e31e6e478dbda3e3f95c6eb306 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:45:36.751930 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f71525e8-97c8-44e7-911d-98475ea2aac3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:36.752714 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-f71525e8-97c8-44e7-911d-98475ea2aac3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:36.754929 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1389/2773] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:45:36 2022] DELETE /identity/v3/projects/188ff1e31e6e478dbda3e3f95c6eb306 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:45:36.760038 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447feae6-7bff-4733-a0ae-440aa76d0ff3 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:36.760564 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447feae6-7bff-4733-a0ae-440aa76d0ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:36.760564 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447feae6-7bff-4733-a0ae-440aa76d0ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:36.787476 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-447feae6-7bff-4733-a0ae-440aa76d0ff3 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:45:36.788685 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-447feae6-7bff-4733-a0ae-440aa76d0ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:36.846962 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1385/2774] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:45:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:45:36.855208 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08586ec-fd80-4c87-ae0b-605afd5faef5 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:36.855610 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08586ec-fd80-4c87-ae0b-605afd5faef5 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:36.855889 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08586ec-fd80-4c87-ae0b-605afd5faef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:36.873901 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-f08586ec-fd80-4c87-ae0b-605afd5faef5 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:45:36.875185 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-f08586ec-fd80-4c87-ae0b-605afd5faef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:36.884061 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1390/2775] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:45:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:45:36.888329 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447feae6-7bff-4733-a0ae-440aa76d0ff3 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:36.893499 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:45:36.899796 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447feae6-7bff-4733-a0ae-440aa76d0ff3 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:36.904395 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad95d7-a193-42d3-bdb7-4e232cd81cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:36.909227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad95d7-a193-42d3-bdb7-4e232cd81cac admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:36.909227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad95d7-a193-42d3-bdb7-4e232cd81cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:36.909227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad95d7-a193-42d3-bdb7-4e232cd81cac admin admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:36.909227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad95d7-a193-42d3-bdb7-4e232cd81cac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:45:36.909227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad95d7-a193-42d3-bdb7-4e232cd81cac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:36.909227 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad95d7-a193-42d3-bdb7-4e232cd81cac admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:36.919960 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1386/2776] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:45:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:45:36.924155 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08586ec-fd80-4c87-ae0b-605afd5faef5 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:36.933471 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08586ec-fd80-4c87-ae0b-605afd5faef5 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:36.938190 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271ab3c-c0aa-4e4e-b870-ab0ee715d941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:36.938839 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271ab3c-c0aa-4e4e-b870-ab0ee715d941 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:36.939127 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271ab3c-c0aa-4e4e-b870-ab0ee715d941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:36.939467 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271ab3c-c0aa-4e4e-b870-ab0ee715d941 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:36.940168 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271ab3c-c0aa-4e4e-b870-ab0ee715d941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:36.942579 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271ab3c-c0aa-4e4e-b870-ab0ee715d941 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:37.102795 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1391/2777] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:45:36 2022] POST /identity/v3/projects => generated 369 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:45:37.106499 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad95d7-a193-42d3-bdb7-4e232cd81cac admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:37.116917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad95d7-a193-42d3-bdb7-4e232cd81cac admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:37.172107 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d0837-0139-493e-9b4c-b1887abe6279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:37.172776 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d0837-0139-493e-9b4c-b1887abe6279 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:37.173062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d0837-0139-493e-9b4c-b1887abe6279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:37.173343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d0837-0139-493e-9b4c-b1887abe6279 admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:37.174026 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d0837-0139-493e-9b4c-b1887abe6279 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:37.176607 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d0837-0139-493e-9b4c-b1887abe6279 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:37.336334 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1387/2778] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:45:37 2022] POST /identity/v3/users => generated 285 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:45:37.340284 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271ab3c-c0aa-4e4e-b870-ab0ee715d941 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:37.351372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271ab3c-c0aa-4e4e-b870-ab0ee715d941 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:37.356030 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee5f35b-5734-4900-a685-2c80c556941c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:37.356675 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5f35b-5734-4900-a685-2c80c556941c admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:37.356964 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5f35b-5734-4900-a685-2c80c556941c admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:37.357245 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee5f35b-5734-4900-a685-2c80c556941c admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:37.357975 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee5f35b-5734-4900-a685-2c80c556941c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:37.359665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee5f35b-5734-4900-a685-2c80c556941c admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:37.370802 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1392/2779] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:45:37.374761 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d0837-0139-493e-9b4c-b1887abe6279 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:37.385182 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d0837-0139-493e-9b4c-b1887abe6279 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:37.389830 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44015cb-c4ef-498c-ac51-d52d7177027b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:37.390492 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44015cb-c4ef-498c-ac51-d52d7177027b admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:37.390781 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44015cb-c4ef-498c-ac51-d52d7177027b admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:37.391061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44015cb-c4ef-498c-ac51-d52d7177027b admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:37.391832 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44015cb-c4ef-498c-ac51-d52d7177027b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:37.393429 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44015cb-c4ef-498c-ac51-d52d7177027b admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:37.403277 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1388/2780] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:45:37.407359 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee5f35b-5734-4900-a685-2c80c556941c admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:37.418302 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee5f35b-5734-4900-a685-2c80c556941c admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:37.422827 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc3105-f6ce-4ab6-98e4-98a7e49bc056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:37.423735 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc3105-f6ce-4ab6-98e4-98a7e49bc056 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:37.424020 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc3105-f6ce-4ab6-98e4-98a7e49bc056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:37.424297 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc3105-f6ce-4ab6-98e4-98a7e49bc056 admin admin] PATH_INFO: `/v3/projects/7acc7d7efd53444db3d14845a7592fa6/users/10604a3f241d4e7ca567bd96a7824f0e/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:37.439086 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc3105-f6ce-4ab6-98e4-98a7e49bc056 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7acc7d7efd53444db3d14845a7592fa6, user_id=10604a3f241d4e7ca567bd96a7824f0e, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:37.441812 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc3105-f6ce-4ab6-98e4-98a7e49bc056 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:37.553323 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1393/2781] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:45:37 2022] PUT /identity/v3/projects/7acc7d7efd53444db3d14845a7592fa6/users/10604a3f241d4e7ca567bd96a7824f0e/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:45:37.570183 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185b494-3391-4b9b-b532-2c4e98e4bd34 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:37.570478 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185b494-3391-4b9b-b532-2c4e98e4bd34 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:37.570761 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185b494-3391-4b9b-b532-2c4e98e4bd34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:37.597041 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-d185b494-3391-4b9b-b532-2c4e98e4bd34 None None] MFA Rules not processed for user `10604a3f241d4e7ca567bd96a7824f0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:45:37.598938 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d185b494-3391-4b9b-b532-2c4e98e4bd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:37.663144 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1389/2782] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:45:37 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:45:37.668206 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc3105-f6ce-4ab6-98e4-98a7e49bc056 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:37.716004 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:45:37.729315 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc3105-f6ce-4ab6-98e4-98a7e49bc056 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:37.734029 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b835c0fa-5eb2-4e36-9ccc-77ff096a1305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:37.734686 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835c0fa-5eb2-4e36-9ccc-77ff096a1305 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:37.734971 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835c0fa-5eb2-4e36-9ccc-77ff096a1305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:37.735250 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b835c0fa-5eb2-4e36-9ccc-77ff096a1305 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:37.735984 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b835c0fa-5eb2-4e36-9ccc-77ff096a1305 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:37.738625 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b835c0fa-5eb2-4e36-9ccc-77ff096a1305 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:37.822440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1394/2783] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:45:37 2022] POST /identity/v3/projects => generated 369 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:45:37.826413 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185b494-3391-4b9b-b532-2c4e98e4bd34 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:37.838773 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185b494-3391-4b9b-b532-2c4e98e4bd34 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:37.888910 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:45:37.895504 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca710a-8291-4da8-9208-950c338747d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:37.896183 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca710a-8291-4da8-9208-950c338747d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:37.896473 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca710a-8291-4da8-9208-950c338747d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:37.896756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ca710a-8291-4da8-9208-950c338747d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:37.897446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca710a-8291-4da8-9208-950c338747d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:37.899397 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ca710a-8291-4da8-9208-950c338747d4 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:37.986768 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1390/2784] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:45:37 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:45:37.991287 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b835c0fa-5eb2-4e36-9ccc-77ff096a1305 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:38.007292 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b835c0fa-5eb2-4e36-9ccc-77ff096a1305 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:38.010677 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf7392-aa03-4505-b10c-f71603805862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:38.011425 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf7392-aa03-4505-b10c-f71603805862 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:38.011716 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf7392-aa03-4505-b10c-f71603805862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:38.011996 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdf7392-aa03-4505-b10c-f71603805862 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:38.012785 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf7392-aa03-4505-b10c-f71603805862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:38.014426 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdf7392-aa03-4505-b10c-f71603805862 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:38.026177 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1395/2785] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:45:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:45:38.030389 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca710a-8291-4da8-9208-950c338747d4 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:38.041921 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ca710a-8291-4da8-9208-950c338747d4 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:38.046501 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7a3c0-4b86-4f9b-82a9-3fdb792a7b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:38.047414 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7a3c0-4b86-4f9b-82a9-3fdb792a7b2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:38.047697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7a3c0-4b86-4f9b-82a9-3fdb792a7b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:38.047972 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7a3c0-4b86-4f9b-82a9-3fdb792a7b2d admin admin] PATH_INFO: `/v3/projects/f42fcc1e2ca64691b602070f0ebd101e/users/dd0c014c89674389a4ddfb9aa5c482c5/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:38.062795 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7a3c0-4b86-4f9b-82a9-3fdb792a7b2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f42fcc1e2ca64691b602070f0ebd101e, user_id=dd0c014c89674389a4ddfb9aa5c482c5, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:38.065462 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7a3c0-4b86-4f9b-82a9-3fdb792a7b2d admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:38.152779 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1391/2786] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:45:38 2022] PUT /identity/v3/projects/f42fcc1e2ca64691b602070f0ebd101e/users/dd0c014c89674389a4ddfb9aa5c482c5/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:45:38.156828 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf7392-aa03-4505-b10c-f71603805862 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:38.167113 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdf7392-aa03-4505-b10c-f71603805862 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:38.222427 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56224204-5a8a-4bec-bd96-de2c02a33b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:38.223103 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56224204-5a8a-4bec-bd96-de2c02a33b13 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:38.223426 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56224204-5a8a-4bec-bd96-de2c02a33b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:38.223713 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56224204-5a8a-4bec-bd96-de2c02a33b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:38.224441 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56224204-5a8a-4bec-bd96-de2c02a33b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:38.226060 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56224204-5a8a-4bec-bd96-de2c02a33b13 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:38.235843 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1396/2787] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:45:38.240118 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7a3c0-4b86-4f9b-82a9-3fdb792a7b2d admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:38.250325 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7a3c0-4b86-4f9b-82a9-3fdb792a7b2d admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:38.255545 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f30882-ca88-4ac6-803c-44f7f6a86448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:38.256430 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f30882-ca88-4ac6-803c-44f7f6a86448 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:38.256713 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f30882-ca88-4ac6-803c-44f7f6a86448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:38.256992 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f30882-ca88-4ac6-803c-44f7f6a86448 admin admin] PATH_INFO: `/v3/projects/f42fcc1e2ca64691b602070f0ebd101e/users/dd0c014c89674389a4ddfb9aa5c482c5/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:38.259012 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f30882-ca88-4ac6-803c-44f7f6a86448 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f42fcc1e2ca64691b602070f0ebd101e, user_id=dd0c014c89674389a4ddfb9aa5c482c5, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:38.261768 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f30882-ca88-4ac6-803c-44f7f6a86448 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:38.272748 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1392/2788] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:45:38 2022] PUT /identity/v3/projects/f42fcc1e2ca64691b602070f0ebd101e/users/dd0c014c89674389a4ddfb9aa5c482c5/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:45:38.334315 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c529da-2688-4f2f-9398-523bdea74384 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:38.334611 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c529da-2688-4f2f-9398-523bdea74384 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:38.334903 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c529da-2688-4f2f-9398-523bdea74384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:38.361318 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-90c529da-2688-4f2f-9398-523bdea74384 None None] MFA Rules not processed for user `dd0c014c89674389a4ddfb9aa5c482c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:45:38.362570 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-90c529da-2688-4f2f-9398-523bdea74384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:38.431365 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1397/2789] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:45:38 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:45:38.443024 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f30882-ca88-4ac6-803c-44f7f6a86448 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:38.444767 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-f8f30882-ca88-4ac6-803c-44f7f6a86448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:38.526778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f30882-ca88-4ac6-803c-44f7f6a86448 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:38.530425 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a93d4-9157-425b-82a9-942a215ef984 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:38.531080 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a93d4-9157-425b-82a9-942a215ef984 service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:38.531404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a93d4-9157-425b-82a9-942a215ef984 service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:38.531690 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a93d4-9157-425b-82a9-942a215ef984 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:38.540999 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a93d4-9157-425b-82a9-942a215ef984 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:38.542431 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a93d4-9157-425b-82a9-942a215ef984 service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:38.552594 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1393/2790] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:45:38 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:45:38.693132 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c529da-2688-4f2f-9398-523bdea74384 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:38.695949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-90c529da-2688-4f2f-9398-523bdea74384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:38.706411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c529da-2688-4f2f-9398-523bdea74384 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:38.709975 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92418692-684d-4d59-a3fa-0f2bcd29f07d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:38.710617 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92418692-684d-4d59-a3fa-0f2bcd29f07d service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:38.710904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92418692-684d-4d59-a3fa-0f2bcd29f07d service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:38.711182 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92418692-684d-4d59-a3fa-0f2bcd29f07d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:38.756130 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:45:38.771288 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92418692-684d-4d59-a3fa-0f2bcd29f07d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:38.774208 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92418692-684d-4d59-a3fa-0f2bcd29f07d service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:38.787084 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1398/2791] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:45:38 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:45:44.458073 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a93d4-9157-425b-82a9-942a215ef984 service cinder] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:44.521081 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a93d4-9157-425b-82a9-942a215ef984 service cinder] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:44.525670 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f05b41-0c16-4ed6-bd48-a317fd6e5e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:44.526559 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f05b41-0c16-4ed6-bd48-a317fd6e5e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:44.526844 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f05b41-0c16-4ed6-bd48-a317fd6e5e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:44.527123 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f05b41-0c16-4ed6-bd48-a317fd6e5e59 admin admin] PATH_INFO: `/v3/users/10604a3f241d4e7ca567bd96a7824f0e` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:44.528367 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f05b41-0c16-4ed6-bd48-a317fd6e5e59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10604a3f241d4e7ca567bd96a7824f0e)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:44.530292 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f05b41-0c16-4ed6-bd48-a317fd6e5e59 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:44.728241 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:45:44.733859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d4f05b41-0c16-4ed6-bd48-a317fd6e5e59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10604a3f241d4e7ca567bd96a7824f0e'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:44.826470 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d4f05b41-0c16-4ed6-bd48-a317fd6e5e59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10604a3f241d4e7ca567bd96a7824f0e'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:44.839090 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d4f05b41-0c16-4ed6-bd48-a317fd6e5e59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10604a3f241d4e7ca567bd96a7824f0e'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:44.839804 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d4f05b41-0c16-4ed6-bd48-a317fd6e5e59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10604a3f241d4e7ca567bd96a7824f0e'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:44.840448 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d4f05b41-0c16-4ed6-bd48-a317fd6e5e59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10604a3f241d4e7ca567bd96a7824f0e'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:44.852907 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1394/2792] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:45:44 2022] DELETE /identity/v3/users/10604a3f241d4e7ca567bd96a7824f0e => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:45:44.857414 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29499fdd-ed23-46ba-b2cd-a1191bfc499b None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:44.857894 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29499fdd-ed23-46ba-b2cd-a1191bfc499b None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:44.858008 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29499fdd-ed23-46ba-b2cd-a1191bfc499b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:44.875232 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-29499fdd-ed23-46ba-b2cd-a1191bfc499b None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:45:44.876507 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-29499fdd-ed23-46ba-b2cd-a1191bfc499b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:44.950676 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1399/2793] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:45:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:45:44.961084 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f05b41-0c16-4ed6-bd48-a317fd6e5e59 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:44.962506 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-d4f05b41-0c16-4ed6-bd48-a317fd6e5e59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:45.041412 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f05b41-0c16-4ed6-bd48-a317fd6e5e59 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:45.045113 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98082d5c-1f2e-4317-9760-dbbc745935ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:45.045756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98082d5c-1f2e-4317-9760-dbbc745935ac service neutron] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:45.046045 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98082d5c-1f2e-4317-9760-dbbc745935ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:45.046324 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98082d5c-1f2e-4317-9760-dbbc745935ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:45.055443 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98082d5c-1f2e-4317-9760-dbbc745935ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:45.056877 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98082d5c-1f2e-4317-9760-dbbc745935ac service neutron] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:45.066925 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1395/2794] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:45:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:45:45.679810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29499fdd-ed23-46ba-b2cd-a1191bfc499b None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:45.681336 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-29499fdd-ed23-46ba-b2cd-a1191bfc499b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:45.694232 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29499fdd-ed23-46ba-b2cd-a1191bfc499b None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:45.700561 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aadb54d-114d-41c9-b244-3e0cd75a2ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:45.700561 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadb54d-114d-41c9-b244-3e0cd75a2ec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:45.700561 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadb54d-114d-41c9-b244-3e0cd75a2ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:45.700561 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aadb54d-114d-41c9-b244-3e0cd75a2ec2 admin admin] PATH_INFO: `/v3/projects/7acc7d7efd53444db3d14845a7592fa6` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:45.701738 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aadb54d-114d-41c9-b244-3e0cd75a2ec2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7acc7d7efd53444db3d14845a7592fa6)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:45.703839 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aadb54d-114d-41c9-b244-3e0cd75a2ec2 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:45.843755 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4aadb54d-114d-41c9-b244-3e0cd75a2ec2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7acc7d7efd53444db3d14845a7592fa6', 'request_id': 'req-4aadb54d-114d-41c9-b244-3e0cd75a2ec2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:45.934933 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4aadb54d-114d-41c9-b244-3e0cd75a2ec2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7acc7d7efd53444db3d14845a7592fa6', 'request_id': 'req-4aadb54d-114d-41c9-b244-3e0cd75a2ec2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:45.940876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4aadb54d-114d-41c9-b244-3e0cd75a2ec2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7acc7d7efd53444db3d14845a7592fa6', 'request_id': 'req-4aadb54d-114d-41c9-b244-3e0cd75a2ec2'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:45.952372 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4aadb54d-114d-41c9-b244-3e0cd75a2ec2 admin admin] The token cache is being invalidate because project 7acc7d7efd53444db3d14845a7592fa6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:45:45.952652 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4aadb54d-114d-41c9-b244-3e0cd75a2ec2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:45.953296 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-4aadb54d-114d-41c9-b244-3e0cd75a2ec2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:45.954905 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1400/2795] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:45:45 2022] DELETE /identity/v3/projects/7acc7d7efd53444db3d14845a7592fa6 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:45:45.959008 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98082d5c-1f2e-4317-9760-dbbc745935ac service neutron] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:45.968824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98082d5c-1f2e-4317-9760-dbbc745935ac service neutron] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:46.024558 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da59d0-efcb-4b60-a19d-cbaab4239f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:46.025446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da59d0-efcb-4b60-a19d-cbaab4239f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:46.025731 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da59d0-efcb-4b60-a19d-cbaab4239f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:46.026010 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da59d0-efcb-4b60-a19d-cbaab4239f5c admin admin] PATH_INFO: `/v3/users/dd0c014c89674389a4ddfb9aa5c482c5` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:46.027206 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da59d0-efcb-4b60-a19d-cbaab4239f5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd0c014c89674389a4ddfb9aa5c482c5)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:46.029154 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da59d0-efcb-4b60-a19d-cbaab4239f5c admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:46.294270 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:45:46.307827 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-04da59d0-efcb-4b60-a19d-cbaab4239f5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd0c014c89674389a4ddfb9aa5c482c5'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:46.390055 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-04da59d0-efcb-4b60-a19d-cbaab4239f5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd0c014c89674389a4ddfb9aa5c482c5'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:46.402528 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-04da59d0-efcb-4b60-a19d-cbaab4239f5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd0c014c89674389a4ddfb9aa5c482c5'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:46.403195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-04da59d0-efcb-4b60-a19d-cbaab4239f5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd0c014c89674389a4ddfb9aa5c482c5'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:46.403828 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-04da59d0-efcb-4b60-a19d-cbaab4239f5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd0c014c89674389a4ddfb9aa5c482c5'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:46.417317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1396/2796] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:45:45 2022] DELETE /identity/v3/users/dd0c014c89674389a4ddfb9aa5c482c5 => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:45:46.942163 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aadb54d-114d-41c9-b244-3e0cd75a2ec2 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:46.943781 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-4aadb54d-114d-41c9-b244-3e0cd75a2ec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:46.958187 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aadb54d-114d-41c9-b244-3e0cd75a2ec2 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:47.016219 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1d384-4f23-42ba-881f-5033bade01a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:47.017091 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1d384-4f23-42ba-881f-5033bade01a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:47.017385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1d384-4f23-42ba-881f-5033bade01a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:47.017670 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1d384-4f23-42ba-881f-5033bade01a7 admin admin] PATH_INFO: `/v3/projects/f42fcc1e2ca64691b602070f0ebd101e` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:47.019435 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf1d384-4f23-42ba-881f-5033bade01a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f42fcc1e2ca64691b602070f0ebd101e)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:47.021454 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf1d384-4f23-42ba-881f-5033bade01a7 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:47.029606 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:45:47.152473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fdf1d384-4f23-42ba-881f-5033bade01a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f42fcc1e2ca64691b602070f0ebd101e', 'request_id': 'req-fdf1d384-4f23-42ba-881f-5033bade01a7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:47.238592 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fdf1d384-4f23-42ba-881f-5033bade01a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f42fcc1e2ca64691b602070f0ebd101e', 'request_id': 'req-fdf1d384-4f23-42ba-881f-5033bade01a7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:47.244794 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fdf1d384-4f23-42ba-881f-5033bade01a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f42fcc1e2ca64691b602070f0ebd101e', 'request_id': 'req-fdf1d384-4f23-42ba-881f-5033bade01a7'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:47.257463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fdf1d384-4f23-42ba-881f-5033bade01a7 admin admin] The token cache is being invalidate because project f42fcc1e2ca64691b602070f0ebd101e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80363) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:45:47.257740 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fdf1d384-4f23-42ba-881f-5033bade01a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:47.258450 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fdf1d384-4f23-42ba-881f-5033bade01a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:45:47.260187 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1401/2797] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:45:46 2022] DELETE /identity/v3/projects/f42fcc1e2ca64691b602070f0ebd101e => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:45:47.265763 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a132235-ad87-45f9-92f3-d42cf9ebc1d2 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:47.266253 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a132235-ad87-45f9-92f3-d42cf9ebc1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:47.266371 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a132235-ad87-45f9-92f3-d42cf9ebc1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:47.304201 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-5a132235-ad87-45f9-92f3-d42cf9ebc1d2 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:45:47.305440 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-5a132235-ad87-45f9-92f3-d42cf9ebc1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:47.365050 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1397/2798] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:45:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:45:47.373015 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3f19f-95bb-4d78-bdbd-bf442b180d86 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:47.373307 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3f19f-95bb-4d78-bdbd-bf442b180d86 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:47.373591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3f19f-95bb-4d78-bdbd-bf442b180d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:47.392261 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-9bc3f19f-95bb-4d78-bdbd-bf442b180d86 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:45:47.393470 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-9bc3f19f-95bb-4d78-bdbd-bf442b180d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:47.401986 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1402/2799] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:45:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:45:47.406259 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a132235-ad87-45f9-92f3-d42cf9ebc1d2 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:47.416404 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a132235-ad87-45f9-92f3-d42cf9ebc1d2 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:47.421457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba287c-c22a-4f80-ba21-086ee005d4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:47.422474 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba287c-c22a-4f80-ba21-086ee005d4a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:47.423011 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba287c-c22a-4f80-ba21-086ee005d4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:47.423511 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba287c-c22a-4f80-ba21-086ee005d4a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:47.424742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba287c-c22a-4f80-ba21-086ee005d4a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80364) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:45:47.425273 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba287c-c22a-4f80-ba21-086ee005d4a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:47.428055 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba287c-c22a-4f80-ba21-086ee005d4a0 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:47.441870 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1398/2800] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:45:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:45:47.446232 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3f19f-95bb-4d78-bdbd-bf442b180d86 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:47.455516 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3f19f-95bb-4d78-bdbd-bf442b180d86 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:47.460907 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a632a-c9ac-4178-8a55-6d11a48b44f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:47.461561 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a632a-c9ac-4178-8a55-6d11a48b44f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:47.461845 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a632a-c9ac-4178-8a55-6d11a48b44f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:47.462123 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a632a-c9ac-4178-8a55-6d11a48b44f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:47.462810 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a632a-c9ac-4178-8a55-6d11a48b44f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:47.464755 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a632a-c9ac-4178-8a55-6d11a48b44f4 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:47.560313 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1403/2801] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:45:47 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:45:47.567182 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba287c-c22a-4f80-ba21-086ee005d4a0 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:47.577224 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba287c-c22a-4f80-ba21-086ee005d4a0 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:47.639744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa442c-6675-4dd2-9347-3562601289c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:47.640421 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa442c-6675-4dd2-9347-3562601289c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:47.640706 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa442c-6675-4dd2-9347-3562601289c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:47.640985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa442c-6675-4dd2-9347-3562601289c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:47.641671 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa442c-6675-4dd2-9347-3562601289c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:47.643617 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa442c-6675-4dd2-9347-3562601289c9 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:47.735329 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1399/2802] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:45:47 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:45:47.737287 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a632a-c9ac-4178-8a55-6d11a48b44f4 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:47.748491 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a632a-c9ac-4178-8a55-6d11a48b44f4 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:47.753812 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6fb4c-f302-4c2d-a22d-a4ece3756854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:47.754489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6fb4c-f302-4c2d-a22d-a4ece3756854 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:47.754775 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6fb4c-f302-4c2d-a22d-a4ece3756854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:47.755054 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb6fb4c-f302-4c2d-a22d-a4ece3756854 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:47.755883 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb6fb4c-f302-4c2d-a22d-a4ece3756854 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:47.757442 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb6fb4c-f302-4c2d-a22d-a4ece3756854 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:47.768205 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1404/2803] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:45:47.772072 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa442c-6675-4dd2-9347-3562601289c9 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:47.782219 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa442c-6675-4dd2-9347-3562601289c9 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:47.786968 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0593e997-3fae-46d4-9770-b95958155667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:47.787654 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593e997-3fae-46d4-9770-b95958155667 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:47.787941 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593e997-3fae-46d4-9770-b95958155667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:47.788217 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0593e997-3fae-46d4-9770-b95958155667 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:47.788942 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593e997-3fae-46d4-9770-b95958155667 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:47.790797 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0593e997-3fae-46d4-9770-b95958155667 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:47.801275 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1400/2804] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:45:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:45:47.805291 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6fb4c-f302-4c2d-a22d-a4ece3756854 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:47.815183 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb6fb4c-f302-4c2d-a22d-a4ece3756854 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:47.820481 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a698533-575c-4b39-9971-c44e197ecdc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:47.821351 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a698533-575c-4b39-9971-c44e197ecdc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:47.821640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a698533-575c-4b39-9971-c44e197ecdc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:47.821919 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a698533-575c-4b39-9971-c44e197ecdc9 admin admin] PATH_INFO: `/v3/projects/00c0c5197cd246058ff502c5221f5f44/users/88b5e9807f3a4bf3a2d078b03d9c9654/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:47.836062 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a698533-575c-4b39-9971-c44e197ecdc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00c0c5197cd246058ff502c5221f5f44, user_id=88b5e9807f3a4bf3a2d078b03d9c9654, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:47.838761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a698533-575c-4b39-9971-c44e197ecdc9 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:47.923680 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1405/2805] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:45:47 2022] PUT /identity/v3/projects/00c0c5197cd246058ff502c5221f5f44/users/88b5e9807f3a4bf3a2d078b03d9c9654/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:45:47.941884 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4ad85-b79c-435b-9e58-b46f804914ba None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:47.942186 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4ad85-b79c-435b-9e58-b46f804914ba None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:47.942474 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed4ad85-b79c-435b-9e58-b46f804914ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:47.960824 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:45:47.969764 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-6ed4ad85-b79c-435b-9e58-b46f804914ba None None] MFA Rules not processed for user `88b5e9807f3a4bf3a2d078b03d9c9654`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:45:47.971033 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-6ed4ad85-b79c-435b-9e58-b46f804914ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:48.036396 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1401/2806] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:45:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:45:48.041157 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a698533-575c-4b39-9971-c44e197ecdc9 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:48.102328 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a698533-575c-4b39-9971-c44e197ecdc9 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:48.110893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb82316c-43f9-40e3-b9dc-7b97057bf31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:48.110893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82316c-43f9-40e3-b9dc-7b97057bf31a admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:48.110893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82316c-43f9-40e3-b9dc-7b97057bf31a admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:48.110893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb82316c-43f9-40e3-b9dc-7b97057bf31a admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:48.111382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb82316c-43f9-40e3-b9dc-7b97057bf31a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:48.111382 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb82316c-43f9-40e3-b9dc-7b97057bf31a admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:48.234006 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1406/2807] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:45:48 2022] POST /identity/v3/projects => generated 385 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:45:48.238283 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4ad85-b79c-435b-9e58-b46f804914ba None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:48.249411 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed4ad85-b79c-435b-9e58-b46f804914ba None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:48.314449 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca67cf-c546-47c7-a787-a70b819bd1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:48.315393 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca67cf-c546-47c7-a787-a70b819bd1bd admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:48.315836 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca67cf-c546-47c7-a787-a70b819bd1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:48.316255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca67cf-c546-47c7-a787-a70b819bd1bd admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:48.317339 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca67cf-c546-47c7-a787-a70b819bd1bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:48.320456 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca67cf-c546-47c7-a787-a70b819bd1bd admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:48.466830 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1402/2808] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:45:48 2022] POST /identity/v3/users => generated 293 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:45:48.470837 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb82316c-43f9-40e3-b9dc-7b97057bf31a admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:48.482324 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb82316c-43f9-40e3-b9dc-7b97057bf31a admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:48.499162 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7711c-6ab2-4e92-92f2-4033ae6812ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:48.499162 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7711c-6ab2-4e92-92f2-4033ae6812ee admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:48.499162 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7711c-6ab2-4e92-92f2-4033ae6812ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:48.499162 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7711c-6ab2-4e92-92f2-4033ae6812ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:48.499629 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c7711c-6ab2-4e92-92f2-4033ae6812ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:48.499629 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c7711c-6ab2-4e92-92f2-4033ae6812ee admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:48.501419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1407/2809] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:45:48.505443 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca67cf-c546-47c7-a787-a70b819bd1bd admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:48.515436 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca67cf-c546-47c7-a787-a70b819bd1bd admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:48.520691 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe54be6-0970-42f6-a5f2-4b7e8fde6b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:48.521350 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe54be6-0970-42f6-a5f2-4b7e8fde6b26 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:48.521635 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe54be6-0970-42f6-a5f2-4b7e8fde6b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:48.521917 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe54be6-0970-42f6-a5f2-4b7e8fde6b26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:48.522676 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe54be6-0970-42f6-a5f2-4b7e8fde6b26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:48.524308 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe54be6-0970-42f6-a5f2-4b7e8fde6b26 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:48.534143 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1403/2810] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:45:48.539630 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7711c-6ab2-4e92-92f2-4033ae6812ee admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:48.549456 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7711c-6ab2-4e92-92f2-4033ae6812ee admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:48.554312 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6146a13-b6c0-44fa-8108-ae7c6e78f36d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:48.555194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6146a13-b6c0-44fa-8108-ae7c6e78f36d admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:48.555528 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6146a13-b6c0-44fa-8108-ae7c6e78f36d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:48.555808 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6146a13-b6c0-44fa-8108-ae7c6e78f36d admin admin] PATH_INFO: `/v3/projects/9349bcae7a8a4393a04f4507234684ee/users/d6788f0566f2484c868a59d3e45e049e/roles/31034f95ef88402296c753c2c259f717` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:48.558073 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:45:48.571222 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6146a13-b6c0-44fa-8108-ae7c6e78f36d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9349bcae7a8a4393a04f4507234684ee, user_id=d6788f0566f2484c868a59d3e45e049e, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:48.573876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6146a13-b6c0-44fa-8108-ae7c6e78f36d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:48.704704 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1408/2811] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:45:48 2022] PUT /identity/v3/projects/9349bcae7a8a4393a04f4507234684ee/users/d6788f0566f2484c868a59d3e45e049e/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:45:48.716790 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674092ad-e6b6-46ad-9c0f-cfbf6622fb06 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:48.717081 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674092ad-e6b6-46ad-9c0f-cfbf6622fb06 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:48.717384 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674092ad-e6b6-46ad-9c0f-cfbf6622fb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:48.743011 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-674092ad-e6b6-46ad-9c0f-cfbf6622fb06 None None] MFA Rules not processed for user `d6788f0566f2484c868a59d3e45e049e`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:45:48.744254 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-674092ad-e6b6-46ad-9c0f-cfbf6622fb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:48.807692 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1404/2812] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:45:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:45:48.812094 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6146a13-b6c0-44fa-8108-ae7c6e78f36d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:48.873275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6146a13-b6c0-44fa-8108-ae7c6e78f36d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:48.877874 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf8254-6cd3-41bd-8bb3-9261297f7445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:48.879123 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf8254-6cd3-41bd-8bb3-9261297f7445 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:48.879458 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf8254-6cd3-41bd-8bb3-9261297f7445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:48.879738 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf8254-6cd3-41bd-8bb3-9261297f7445 admin admin] PATH_INFO: `/v3/projects` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:48.880440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf8254-6cd3-41bd-8bb3-9261297f7445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:48.882356 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf8254-6cd3-41bd-8bb3-9261297f7445 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:49.006414 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1409/2813] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:45:48 2022] POST /identity/v3/projects => generated 385 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:45:49.010270 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674092ad-e6b6-46ad-9c0f-cfbf6622fb06 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:49.021075 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674092ad-e6b6-46ad-9c0f-cfbf6622fb06 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:49.025037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:45:49.080098 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423daa83-0f9f-4228-a741-d1430af17ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:49.080775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423daa83-0f9f-4228-a741-d1430af17ece admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:49.081057 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423daa83-0f9f-4228-a741-d1430af17ece admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:49.081579 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423daa83-0f9f-4228-a741-d1430af17ece admin admin] PATH_INFO: `/v3/users` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:49.082036 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423daa83-0f9f-4228-a741-d1430af17ece admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:49.083986 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423daa83-0f9f-4228-a741-d1430af17ece admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:49.245046 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1405/2814] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:45:49 2022] POST /identity/v3/users => generated 299 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:45:49.248837 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf8254-6cd3-41bd-8bb3-9261297f7445 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:49.259826 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf8254-6cd3-41bd-8bb3-9261297f7445 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:49.264361 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58546a-ce9a-4cc9-993b-afd8a44d8499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:49.265135 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58546a-ce9a-4cc9-993b-afd8a44d8499 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:49.265354 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58546a-ce9a-4cc9-993b-afd8a44d8499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:49.265634 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a58546a-ce9a-4cc9-993b-afd8a44d8499 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:49.266362 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58546a-ce9a-4cc9-993b-afd8a44d8499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:49.268027 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a58546a-ce9a-4cc9-993b-afd8a44d8499 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:49.278707 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1410/2815] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:45:49.297951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423daa83-0f9f-4228-a741-d1430af17ece admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:49.308847 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423daa83-0f9f-4228-a741-d1430af17ece admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:49.313413 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7668d86-ab74-4448-94e6-392fe9248429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:49.314294 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7668d86-ab74-4448-94e6-392fe9248429 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:49.314576 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7668d86-ab74-4448-94e6-392fe9248429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:49.315121 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7668d86-ab74-4448-94e6-392fe9248429 admin admin] PATH_INFO: `/v3/projects/c63b986bb1724b8684cd1d5f7b9fa03b/users/992a67d74dd746c59a22c62e720b6bd0/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:49.332090 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7668d86-ab74-4448-94e6-392fe9248429 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c63b986bb1724b8684cd1d5f7b9fa03b, user_id=992a67d74dd746c59a22c62e720b6bd0, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:49.363856 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7668d86-ab74-4448-94e6-392fe9248429 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:49.530231 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1406/2816] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:45:49 2022] PUT /identity/v3/projects/c63b986bb1724b8684cd1d5f7b9fa03b/users/992a67d74dd746c59a22c62e720b6bd0/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:45:49.539700 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58546a-ce9a-4cc9-993b-afd8a44d8499 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:49.545366 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a58546a-ce9a-4cc9-993b-afd8a44d8499 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:49.601790 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444dd8d0-6cb8-4ded-92a8-38789e5d2ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:49.602463 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dd8d0-6cb8-4ded-92a8-38789e5d2ade admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:49.602748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dd8d0-6cb8-4ded-92a8-38789e5d2ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:49.603026 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444dd8d0-6cb8-4ded-92a8-38789e5d2ade admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:49.603783 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444dd8d0-6cb8-4ded-92a8-38789e5d2ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:49.605385 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444dd8d0-6cb8-4ded-92a8-38789e5d2ade admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:49.615675 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1411/2817] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:45:49 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:45:49.619896 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7668d86-ab74-4448-94e6-392fe9248429 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:49.630622 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7668d86-ab74-4448-94e6-392fe9248429 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:49.635153 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb29070-9fab-4451-aeda-02cc8772e53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:49.636069 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb29070-9fab-4451-aeda-02cc8772e53f admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:49.636351 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb29070-9fab-4451-aeda-02cc8772e53f admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:49.636624 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb29070-9fab-4451-aeda-02cc8772e53f admin admin] PATH_INFO: `/v3/projects/c63b986bb1724b8684cd1d5f7b9fa03b/users/992a67d74dd746c59a22c62e720b6bd0/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:49.638582 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb29070-9fab-4451-aeda-02cc8772e53f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c63b986bb1724b8684cd1d5f7b9fa03b, user_id=992a67d74dd746c59a22c62e720b6bd0, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:49.641223 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb29070-9fab-4451-aeda-02cc8772e53f admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:49.651996 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1407/2818] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:45:49 2022] PUT /identity/v3/projects/c63b986bb1724b8684cd1d5f7b9fa03b/users/992a67d74dd746c59a22c62e720b6bd0/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:45:49.662535 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53543ea-59ab-44b8-b3cb-80f3d10e307d None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:49.662824 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53543ea-59ab-44b8-b3cb-80f3d10e307d None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:49.663099 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53543ea-59ab-44b8-b3cb-80f3d10e307d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:49.689987 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-d53543ea-59ab-44b8-b3cb-80f3d10e307d None None] MFA Rules not processed for user `992a67d74dd746c59a22c62e720b6bd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:45:49.691378 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d53543ea-59ab-44b8-b3cb-80f3d10e307d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:49.734883 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:45:49.761476 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1412/2819] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:45:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:45:49.773230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb29070-9fab-4451-aeda-02cc8772e53f admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:49.774913 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-bcb29070-9fab-4451-aeda-02cc8772e53f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:49.833101 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb29070-9fab-4451-aeda-02cc8772e53f admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:49.836910 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f4bf5c-f225-47cd-a75e-86894d419672 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:49.837562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4bf5c-f225-47cd-a75e-86894d419672 service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:49.837857 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4bf5c-f225-47cd-a75e-86894d419672 service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:49.838140 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f4bf5c-f225-47cd-a75e-86894d419672 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:49.847963 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f4bf5c-f225-47cd-a75e-86894d419672 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:49.849407 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f4bf5c-f225-47cd-a75e-86894d419672 service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:49.858834 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1408/2820] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:45:49 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:45:51.106917 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53543ea-59ab-44b8-b3cb-80f3d10e307d None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:51.108355 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-d53543ea-59ab-44b8-b3cb-80f3d10e307d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:45:51.121101 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53543ea-59ab-44b8-b3cb-80f3d10e307d None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:51.124665 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf92881-14ed-4f48-b212-12c3d32f50b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:51.125309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf92881-14ed-4f48-b212-12c3d32f50b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:51.125602 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf92881-14ed-4f48-b212-12c3d32f50b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:51.125883 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf92881-14ed-4f48-b212-12c3d32f50b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:51.182904 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf92881-14ed-4f48-b212-12c3d32f50b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:51.184450 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf92881-14ed-4f48-b212-12c3d32f50b9 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:51.193563 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1413/2821] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:45:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:45:57.870880 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f4bf5c-f225-47cd-a75e-86894d419672 service cinder] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:45:57.879627 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f4bf5c-f225-47cd-a75e-86894d419672 service cinder] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:45:57.883485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26f3b63-235d-4837-a2b8-538f8633a050 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:45:57.884119 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f3b63-235d-4837-a2b8-538f8633a050 service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:45:57.884406 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f3b63-235d-4837-a2b8-538f8633a050 service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:45:57.884686 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26f3b63-235d-4837-a2b8-538f8633a050 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:45:57.942451 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f3b63-235d-4837-a2b8-538f8633a050 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:45:57.944041 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26f3b63-235d-4837-a2b8-538f8633a050 service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:45:57.948813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:45:57.953504 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1409/2822] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:45:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:46:01.189228 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=38 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:46:02.772781 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf92881-14ed-4f48-b212-12c3d32f50b9 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:02.834317 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf92881-14ed-4f48-b212-12c3d32f50b9 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:02.838852 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cac4b9-0144-415c-ae40-e91e7443b127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:02.839749 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cac4b9-0144-415c-ae40-e91e7443b127 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:02.840036 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cac4b9-0144-415c-ae40-e91e7443b127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:02.840317 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cac4b9-0144-415c-ae40-e91e7443b127 admin admin] PATH_INFO: `/v3/users/88b5e9807f3a4bf3a2d078b03d9c9654` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:02.841506 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cac4b9-0144-415c-ae40-e91e7443b127 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88b5e9807f3a4bf3a2d078b03d9c9654)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:02.843484 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cac4b9-0144-415c-ae40-e91e7443b127 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:03.045798 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c0cac4b9-0144-415c-ae40-e91e7443b127 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88b5e9807f3a4bf3a2d078b03d9c9654'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:03.046386 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c0cac4b9-0144-415c-ae40-e91e7443b127 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88b5e9807f3a4bf3a2d078b03d9c9654'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:03.053745 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c0cac4b9-0144-415c-ae40-e91e7443b127 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88b5e9807f3a4bf3a2d078b03d9c9654'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:03.054383 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c0cac4b9-0144-415c-ae40-e91e7443b127 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88b5e9807f3a4bf3a2d078b03d9c9654'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:03.134501 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-c0cac4b9-0144-415c-ae40-e91e7443b127 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88b5e9807f3a4bf3a2d078b03d9c9654'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:03.153192 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1414/2823] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:46:02 2022] DELETE /identity/v3/users/88b5e9807f3a4bf3a2d078b03d9c9654 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:46:03.157778 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d481f-a792-445b-a6ba-73f895f01560 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:03.158248 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d481f-a792-445b-a6ba-73f895f01560 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:03.158375 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d481f-a792-445b-a6ba-73f895f01560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:03.175362 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-4a0d481f-a792-445b-a6ba-73f895f01560 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:46:03.176551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-4a0d481f-a792-445b-a6ba-73f895f01560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:03.234215 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1410/2824] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:46:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:46:03.244454 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cac4b9-0144-415c-ae40-e91e7443b127 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:03.245949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c0cac4b9-0144-415c-ae40-e91e7443b127 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:03.304060 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cac4b9-0144-415c-ae40-e91e7443b127 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:03.307610 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe075b82-3bd0-4ddc-a76e-92ebecfb2c2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:03.308253 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe075b82-3bd0-4ddc-a76e-92ebecfb2c2d service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:03.308536 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe075b82-3bd0-4ddc-a76e-92ebecfb2c2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:03.308814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe075b82-3bd0-4ddc-a76e-92ebecfb2c2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:03.317856 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe075b82-3bd0-4ddc-a76e-92ebecfb2c2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:03.320370 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe075b82-3bd0-4ddc-a76e-92ebecfb2c2d service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:03.329737 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1415/2825] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:46:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:46:03.751304 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d481f-a792-445b-a6ba-73f895f01560 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:03.752744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-4a0d481f-a792-445b-a6ba-73f895f01560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:03.766908 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d481f-a792-445b-a6ba-73f895f01560 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:03.773839 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb109cd-11b7-46d4-bbbc-4613ba38173c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:03.774873 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb109cd-11b7-46d4-bbbc-4613ba38173c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:03.775327 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb109cd-11b7-46d4-bbbc-4613ba38173c admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:03.776150 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb109cd-11b7-46d4-bbbc-4613ba38173c admin admin] PATH_INFO: `/v3/projects/00c0c5197cd246058ff502c5221f5f44` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:03.777742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb109cd-11b7-46d4-bbbc-4613ba38173c admin admin] RBAC: Authorizing `identity:delete_project(project_id=00c0c5197cd246058ff502c5221f5f44)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:03.779969 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb109cd-11b7-46d4-bbbc-4613ba38173c admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:03.961816 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-7fb109cd-11b7-46d4-bbbc-4613ba38173c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00c0c5197cd246058ff502c5221f5f44', 'request_id': 'req-7fb109cd-11b7-46d4-bbbc-4613ba38173c'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:03.967993 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-7fb109cd-11b7-46d4-bbbc-4613ba38173c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00c0c5197cd246058ff502c5221f5f44', 'request_id': 'req-7fb109cd-11b7-46d4-bbbc-4613ba38173c'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:03.973447 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-7fb109cd-11b7-46d4-bbbc-4613ba38173c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00c0c5197cd246058ff502c5221f5f44', 'request_id': 'req-7fb109cd-11b7-46d4-bbbc-4613ba38173c'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:04.063132 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-7fb109cd-11b7-46d4-bbbc-4613ba38173c admin admin] The token cache is being invalidate because project 00c0c5197cd246058ff502c5221f5f44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:46:04.063450 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-7fb109cd-11b7-46d4-bbbc-4613ba38173c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:04.064082 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-7fb109cd-11b7-46d4-bbbc-4613ba38173c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:04.065637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1411/2826] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:46:03 2022] DELETE /identity/v3/projects/00c0c5197cd246058ff502c5221f5f44 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:46:04.070311 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe075b82-3bd0-4ddc-a76e-92ebecfb2c2d service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:04.080353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe075b82-3bd0-4ddc-a76e-92ebecfb2c2d service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:04.099138 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:46:04.139993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4141e68-4c43-4795-ad30-8a88a7eb3b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:04.143447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4141e68-4c43-4795-ad30-8a88a7eb3b1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:04.143447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4141e68-4c43-4795-ad30-8a88a7eb3b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:04.143447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4141e68-4c43-4795-ad30-8a88a7eb3b1d admin admin] PATH_INFO: `/v3/users/d6788f0566f2484c868a59d3e45e049e` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:04.143447 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4141e68-4c43-4795-ad30-8a88a7eb3b1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6788f0566f2484c868a59d3e45e049e)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:04.144512 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4141e68-4c43-4795-ad30-8a88a7eb3b1d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:04.403915 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a4141e68-4c43-4795-ad30-8a88a7eb3b1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6788f0566f2484c868a59d3e45e049e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:04.404515 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a4141e68-4c43-4795-ad30-8a88a7eb3b1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6788f0566f2484c868a59d3e45e049e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:04.412170 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a4141e68-4c43-4795-ad30-8a88a7eb3b1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6788f0566f2484c868a59d3e45e049e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:04.412823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a4141e68-4c43-4795-ad30-8a88a7eb3b1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6788f0566f2484c868a59d3e45e049e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:04.521430 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a4141e68-4c43-4795-ad30-8a88a7eb3b1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6788f0566f2484c868a59d3e45e049e'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:04.540689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1416/2827] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:46:04 2022] DELETE /identity/v3/users/d6788f0566f2484c868a59d3e45e049e => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:46:04.959551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb109cd-11b7-46d4-bbbc-4613ba38173c admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:04.961029 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-7fb109cd-11b7-46d4-bbbc-4613ba38173c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:04.975446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb109cd-11b7-46d4-bbbc-4613ba38173c admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:05.031806 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8047d4-b5c3-4a1b-a4a7-43002e4eec34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:05.032673 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8047d4-b5c3-4a1b-a4a7-43002e4eec34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:05.032957 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8047d4-b5c3-4a1b-a4a7-43002e4eec34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:05.033235 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8047d4-b5c3-4a1b-a4a7-43002e4eec34 admin admin] PATH_INFO: `/v3/projects/9349bcae7a8a4393a04f4507234684ee` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:05.034419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8047d4-b5c3-4a1b-a4a7-43002e4eec34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9349bcae7a8a4393a04f4507234684ee)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:05.036377 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8047d4-b5c3-4a1b-a4a7-43002e4eec34 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:05.170835 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:46:05.186651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-4b8047d4-b5c3-4a1b-a4a7-43002e4eec34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9349bcae7a8a4393a04f4507234684ee', 'request_id': 'req-4b8047d4-b5c3-4a1b-a4a7-43002e4eec34'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:05.193388 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-4b8047d4-b5c3-4a1b-a4a7-43002e4eec34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9349bcae7a8a4393a04f4507234684ee', 'request_id': 'req-4b8047d4-b5c3-4a1b-a4a7-43002e4eec34'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:05.198914 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-4b8047d4-b5c3-4a1b-a4a7-43002e4eec34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9349bcae7a8a4393a04f4507234684ee', 'request_id': 'req-4b8047d4-b5c3-4a1b-a4a7-43002e4eec34'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:05.350889 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-4b8047d4-b5c3-4a1b-a4a7-43002e4eec34 admin admin] The token cache is being invalidate because project 9349bcae7a8a4393a04f4507234684ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:46:05.351104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-4b8047d4-b5c3-4a1b-a4a7-43002e4eec34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:05.351766 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-4b8047d4-b5c3-4a1b-a4a7-43002e4eec34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:05.353403 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1412/2828] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:46:04 2022] DELETE /identity/v3/projects/9349bcae7a8a4393a04f4507234684ee => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:46:05.357468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4141e68-4c43-4795-ad30-8a88a7eb3b1d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:05.358898 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a4141e68-4c43-4795-ad30-8a88a7eb3b1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:05.371993 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4141e68-4c43-4795-ad30-8a88a7eb3b1d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:05.428804 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eef19c-8df5-4b2e-b90d-504e1ec4b061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:05.429681 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eef19c-8df5-4b2e-b90d-504e1ec4b061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:05.429967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eef19c-8df5-4b2e-b90d-504e1ec4b061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:05.430247 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eef19c-8df5-4b2e-b90d-504e1ec4b061 admin admin] PATH_INFO: `/v3/users/992a67d74dd746c59a22c62e720b6bd0` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:05.431478 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eef19c-8df5-4b2e-b90d-504e1ec4b061 admin admin] RBAC: Authorizing `identity:delete_user(user_id=992a67d74dd746c59a22c62e720b6bd0)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:05.433424 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eef19c-8df5-4b2e-b90d-504e1ec4b061 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:05.683248 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:46:05.685765 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-48eef19c-8df5-4b2e-b90d-504e1ec4b061 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '992a67d74dd746c59a22c62e720b6bd0'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:05.686454 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-48eef19c-8df5-4b2e-b90d-504e1ec4b061 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '992a67d74dd746c59a22c62e720b6bd0'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:05.693834 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-48eef19c-8df5-4b2e-b90d-504e1ec4b061 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '992a67d74dd746c59a22c62e720b6bd0'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:05.694483 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-48eef19c-8df5-4b2e-b90d-504e1ec4b061 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '992a67d74dd746c59a22c62e720b6bd0'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:05.849823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-48eef19c-8df5-4b2e-b90d-504e1ec4b061 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '992a67d74dd746c59a22c62e720b6bd0'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:05.868327 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1417/2829] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:46:05 2022] DELETE /identity/v3/users/992a67d74dd746c59a22c62e720b6bd0 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:46:06.353094 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8047d4-b5c3-4a1b-a4a7-43002e4eec34 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:06.354577 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-4b8047d4-b5c3-4a1b-a4a7-43002e4eec34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:06.368452 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8047d4-b5c3-4a1b-a4a7-43002e4eec34 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:06.428343 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75aff09-0d7a-44c0-b1a0-8d9c6d9d4922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:06.429203 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75aff09-0d7a-44c0-b1a0-8d9c6d9d4922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:06.429487 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75aff09-0d7a-44c0-b1a0-8d9c6d9d4922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:06.429769 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75aff09-0d7a-44c0-b1a0-8d9c6d9d4922 admin admin] PATH_INFO: `/v3/projects/c63b986bb1724b8684cd1d5f7b9fa03b` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:06.430943 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75aff09-0d7a-44c0-b1a0-8d9c6d9d4922 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c63b986bb1724b8684cd1d5f7b9fa03b)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:06.432886 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75aff09-0d7a-44c0-b1a0-8d9c6d9d4922 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:06.569660 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d75aff09-0d7a-44c0-b1a0-8d9c6d9d4922 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c63b986bb1724b8684cd1d5f7b9fa03b', 'request_id': 'req-d75aff09-0d7a-44c0-b1a0-8d9c6d9d4922'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:06.575722 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d75aff09-0d7a-44c0-b1a0-8d9c6d9d4922 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c63b986bb1724b8684cd1d5f7b9fa03b', 'request_id': 'req-d75aff09-0d7a-44c0-b1a0-8d9c6d9d4922'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:06.581115 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d75aff09-0d7a-44c0-b1a0-8d9c6d9d4922 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c63b986bb1724b8684cd1d5f7b9fa03b', 'request_id': 'req-d75aff09-0d7a-44c0-b1a0-8d9c6d9d4922'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:06.671058 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d75aff09-0d7a-44c0-b1a0-8d9c6d9d4922 admin admin] The token cache is being invalidate because project c63b986bb1724b8684cd1d5f7b9fa03b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:46:06.671387 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d75aff09-0d7a-44c0-b1a0-8d9c6d9d4922 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:06.671999 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-d75aff09-0d7a-44c0-b1a0-8d9c6d9d4922 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:06.673737 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1413/2830] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:46:06 2022] DELETE /identity/v3/projects/c63b986bb1724b8684cd1d5f7b9fa03b => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:46:06.680229 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed1991-7f29-493c-b726-a162a0daa971 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:06.680733 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed1991-7f29-493c-b726-a162a0daa971 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:06.680844 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ed1991-7f29-493c-b726-a162a0daa971 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:06.703282 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-95ed1991-7f29-493c-b726-a162a0daa971 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:46:06.704522 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-95ed1991-7f29-493c-b726-a162a0daa971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:06.769315 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1418/2831] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:46:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:46:06.779300 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cacbcc-c7a7-4020-8777-48e479480b5a None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:06.779607 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cacbcc-c7a7-4020-8777-48e479480b5a None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:06.779889 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cacbcc-c7a7-4020-8777-48e479480b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:06.798274 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-18cacbcc-c7a7-4020-8777-48e479480b5a None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:46:06.799503 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-18cacbcc-c7a7-4020-8777-48e479480b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:06.807596 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1414/2832] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:46:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:46:06.811690 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed1991-7f29-493c-b726-a162a0daa971 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:06.822173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ed1991-7f29-493c-b726-a162a0daa971 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:06.826657 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf2022-ebf4-45fd-bdce-f693f8d8f4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:06.827328 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf2022-ebf4-45fd-bdce-f693f8d8f4f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:06.827618 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf2022-ebf4-45fd-bdce-f693f8d8f4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:06.827896 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cf2022-ebf4-45fd-bdce-f693f8d8f4f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:06.828624 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf2022-ebf4-45fd-bdce-f693f8d8f4f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:46:06.828933 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf2022-ebf4-45fd-bdce-f693f8d8f4f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:06.830546 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cf2022-ebf4-45fd-bdce-f693f8d8f4f6 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:06.842539 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1419/2833] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:46:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 30 10:46:06.846732 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cacbcc-c7a7-4020-8777-48e479480b5a None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:06.856165 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cacbcc-c7a7-4020-8777-48e479480b5a None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:06.860800 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb7fa2-349a-4ef6-8b8c-6128a148b665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:06.861437 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb7fa2-349a-4ef6-8b8c-6128a148b665 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:06.861722 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb7fa2-349a-4ef6-8b8c-6128a148b665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:06.861998 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abb7fa2-349a-4ef6-8b8c-6128a148b665 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:06.862754 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb7fa2-349a-4ef6-8b8c-6128a148b665 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:06.864745 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abb7fa2-349a-4ef6-8b8c-6128a148b665 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:06.948359 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1415/2834] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:46:06 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:46:06.952586 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf2022-ebf4-45fd-bdce-f693f8d8f4f6 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:06.962916 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cf2022-ebf4-45fd-bdce-f693f8d8f4f6 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:07.017949 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294a0ba-e2bf-4dab-8447-4195ba3d0b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:07.018609 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294a0ba-e2bf-4dab-8447-4195ba3d0b3e admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:07.018895 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294a0ba-e2bf-4dab-8447-4195ba3d0b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:07.019169 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294a0ba-e2bf-4dab-8447-4195ba3d0b3e admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:07.019878 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294a0ba-e2bf-4dab-8447-4195ba3d0b3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:07.022332 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b294a0ba-e2bf-4dab-8447-4195ba3d0b3e admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:07.129648 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1420/2835] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:46:06 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:46:07.134291 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb7fa2-349a-4ef6-8b8c-6128a148b665 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:07.144548 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abb7fa2-349a-4ef6-8b8c-6128a148b665 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:07.149733 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedef7f5-f1b0-4b40-83bf-cfd8000826d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:07.150390 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedef7f5-f1b0-4b40-83bf-cfd8000826d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:07.150673 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedef7f5-f1b0-4b40-83bf-cfd8000826d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:07.150950 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedef7f5-f1b0-4b40-83bf-cfd8000826d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:07.151703 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedef7f5-f1b0-4b40-83bf-cfd8000826d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:07.153413 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedef7f5-f1b0-4b40-83bf-cfd8000826d8 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:07.164241 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1416/2836] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:46:07.170731 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294a0ba-e2bf-4dab-8447-4195ba3d0b3e admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:07.181556 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294a0ba-e2bf-4dab-8447-4195ba3d0b3e admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:07.186305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6c2234-f1a0-4a32-8481-a1f14b85cfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:07.186959 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c2234-f1a0-4a32-8481-a1f14b85cfd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:07.187246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c2234-f1a0-4a32-8481-a1f14b85cfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:07.187549 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6c2234-f1a0-4a32-8481-a1f14b85cfd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:07.188277 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6c2234-f1a0-4a32-8481-a1f14b85cfd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:07.189922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6c2234-f1a0-4a32-8481-a1f14b85cfd0 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:07.200758 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1421/2837] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:46:07.204654 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedef7f5-f1b0-4b40-83bf-cfd8000826d8 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:07.211704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:46:07.251451 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedef7f5-f1b0-4b40-83bf-cfd8000826d8 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:07.257258 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12243c87-3e13-4ed7-a59b-e86fc448b6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:07.258142 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12243c87-3e13-4ed7-a59b-e86fc448b6f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:07.258422 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12243c87-3e13-4ed7-a59b-e86fc448b6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:07.258697 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12243c87-3e13-4ed7-a59b-e86fc448b6f0 admin admin] PATH_INFO: `/v3/projects/e152d782df4f45fd8e2002887efcb694/users/45e2a0b6de6d48f3944bcf6162cc531f/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:07.273990 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12243c87-3e13-4ed7-a59b-e86fc448b6f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e152d782df4f45fd8e2002887efcb694, user_id=45e2a0b6de6d48f3944bcf6162cc531f, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:07.277023 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12243c87-3e13-4ed7-a59b-e86fc448b6f0 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:07.370825 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1417/2838] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:46:07 2022] PUT /identity/v3/projects/e152d782df4f45fd8e2002887efcb694/users/45e2a0b6de6d48f3944bcf6162cc531f/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:46:07.389598 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e9531-f0d8-46b3-bbc2-e59d030b8e80 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:07.389894 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e9531-f0d8-46b3-bbc2-e59d030b8e80 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:07.390175 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73e9531-f0d8-46b3-bbc2-e59d030b8e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:07.417314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-e73e9531-f0d8-46b3-bbc2-e59d030b8e80 None None] MFA Rules not processed for user `45e2a0b6de6d48f3944bcf6162cc531f`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:46:07.418543 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e73e9531-f0d8-46b3-bbc2-e59d030b8e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:07.431397 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:46:07.482932 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1422/2839] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:46:07 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:46:07.487446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12243c87-3e13-4ed7-a59b-e86fc448b6f0 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:07.549859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12243c87-3e13-4ed7-a59b-e86fc448b6f0 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:07.554648 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b319b0-6f49-429c-bf9a-a7049e673247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:07.555329 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b319b0-6f49-429c-bf9a-a7049e673247 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:07.555621 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b319b0-6f49-429c-bf9a-a7049e673247 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:07.555897 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b319b0-6f49-429c-bf9a-a7049e673247 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:07.556589 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b319b0-6f49-429c-bf9a-a7049e673247 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:07.558485 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b319b0-6f49-429c-bf9a-a7049e673247 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:07.644518 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1418/2840] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:46:07 2022] POST /identity/v3/projects => generated 379 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:46:07.649254 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e9531-f0d8-46b3-bbc2-e59d030b8e80 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:07.660036 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73e9531-f0d8-46b3-bbc2-e59d030b8e80 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:07.716287 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d53393-bda7-401a-8096-122822c70793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:07.716984 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d53393-bda7-401a-8096-122822c70793 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:07.717269 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d53393-bda7-401a-8096-122822c70793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:07.717548 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d53393-bda7-401a-8096-122822c70793 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:07.718242 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d53393-bda7-401a-8096-122822c70793 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:07.720225 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d53393-bda7-401a-8096-122822c70793 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:07.826492 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1423/2841] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:46:07 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:46:07.830702 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b319b0-6f49-429c-bf9a-a7049e673247 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:07.841176 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b319b0-6f49-429c-bf9a-a7049e673247 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:07.845803 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ca721-cff2-4f43-8ee9-9dfe799deb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:07.846457 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ca721-cff2-4f43-8ee9-9dfe799deb5e admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:07.846739 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ca721-cff2-4f43-8ee9-9dfe799deb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:07.847013 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6ca721-cff2-4f43-8ee9-9dfe799deb5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:07.847848 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ca721-cff2-4f43-8ee9-9dfe799deb5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:07.849884 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6ca721-cff2-4f43-8ee9-9dfe799deb5e admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:07.860237 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1419/2842] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:46:07.864090 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d53393-bda7-401a-8096-122822c70793 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:07.874732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d53393-bda7-401a-8096-122822c70793 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:07.879486 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a13e64-88ca-4541-bed9-6d6e3b001778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:07.880217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a13e64-88ca-4541-bed9-6d6e3b001778 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:07.881246 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a13e64-88ca-4541-bed9-6d6e3b001778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:07.881669 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a13e64-88ca-4541-bed9-6d6e3b001778 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:07.882405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a13e64-88ca-4541-bed9-6d6e3b001778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:07.884044 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a13e64-88ca-4541-bed9-6d6e3b001778 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:07.893974 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1424/2843] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:46:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:46:07.898419 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ca721-cff2-4f43-8ee9-9dfe799deb5e admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:07.908330 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6ca721-cff2-4f43-8ee9-9dfe799deb5e admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:07.912967 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b802b580-70de-4e59-bf22-f7ebb8346020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:07.913838 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b802b580-70de-4e59-bf22-f7ebb8346020 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:07.914124 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b802b580-70de-4e59-bf22-f7ebb8346020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:07.914398 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b802b580-70de-4e59-bf22-f7ebb8346020 admin admin] PATH_INFO: `/v3/projects/1f800e23481c4709816443e271322eec/users/9fa423226f964615a648b0ee016efffa/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:07.929529 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b802b580-70de-4e59-bf22-f7ebb8346020 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f800e23481c4709816443e271322eec, user_id=9fa423226f964615a648b0ee016efffa, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:07.932240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b802b580-70de-4e59-bf22-f7ebb8346020 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:08.032502 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1420/2844] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:46:07 2022] PUT /identity/v3/projects/1f800e23481c4709816443e271322eec/users/9fa423226f964615a648b0ee016efffa/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:46:08.043844 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429da7c-85e8-462d-a0a3-cf8e595d84e2 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:08.044140 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429da7c-85e8-462d-a0a3-cf8e595d84e2 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:08.044422 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c429da7c-85e8-462d-a0a3-cf8e595d84e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:08.071069 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-c429da7c-85e8-462d-a0a3-cf8e595d84e2 None None] MFA Rules not processed for user `9fa423226f964615a648b0ee016efffa`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:46:08.072338 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-c429da7c-85e8-462d-a0a3-cf8e595d84e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:08.137344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1425/2845] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:46:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:46:08.141740 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b802b580-70de-4e59-bf22-f7ebb8346020 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:08.202471 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b802b580-70de-4e59-bf22-f7ebb8346020 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:08.207164 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f2f755-bda5-434e-9b41-fd5d0fafa0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:08.207850 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2f755-bda5-434e-9b41-fd5d0fafa0f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:08.208137 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2f755-bda5-434e-9b41-fd5d0fafa0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:08.208444 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2f755-bda5-434e-9b41-fd5d0fafa0f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:08.209135 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f2f755-bda5-434e-9b41-fd5d0fafa0f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:08.211055 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f2f755-bda5-434e-9b41-fd5d0fafa0f8 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:08.295309 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1421/2846] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:46:08 2022] POST /identity/v3/projects => generated 377 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:46:08.299525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429da7c-85e8-462d-a0a3-cf8e595d84e2 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:08.312072 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c429da7c-85e8-462d-a0a3-cf8e595d84e2 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:08.331929 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:46:08.369517 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950e97d-2ff8-49e8-b6a1-4533612b2830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:08.370189 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950e97d-2ff8-49e8-b6a1-4533612b2830 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:08.370473 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950e97d-2ff8-49e8-b6a1-4533612b2830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:08.370754 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6950e97d-2ff8-49e8-b6a1-4533612b2830 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:08.371485 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6950e97d-2ff8-49e8-b6a1-4533612b2830 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:08.373482 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6950e97d-2ff8-49e8-b6a1-4533612b2830 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:08.454876 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1426/2847] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:46:08 2022] POST /identity/v3/users => generated 295 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:46:08.459395 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f2f755-bda5-434e-9b41-fd5d0fafa0f8 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:08.469775 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f2f755-bda5-434e-9b41-fd5d0fafa0f8 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:08.475092 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a625e1-1501-4606-9e54-9be80b8f7f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:08.475799 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a625e1-1501-4606-9e54-9be80b8f7f5b admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:08.476128 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a625e1-1501-4606-9e54-9be80b8f7f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:08.476427 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a625e1-1501-4606-9e54-9be80b8f7f5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:08.477158 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a625e1-1501-4606-9e54-9be80b8f7f5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:08.478759 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a625e1-1501-4606-9e54-9be80b8f7f5b admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:08.489310 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1422/2848] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:46:08.493950 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950e97d-2ff8-49e8-b6a1-4533612b2830 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:08.504075 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6950e97d-2ff8-49e8-b6a1-4533612b2830 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:08.508893 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80fe67f-c7a1-4bf7-846f-3f9943c5b702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:08.509778 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80fe67f-c7a1-4bf7-846f-3f9943c5b702 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:08.510066 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80fe67f-c7a1-4bf7-846f-3f9943c5b702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:08.510344 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80fe67f-c7a1-4bf7-846f-3f9943c5b702 admin admin] PATH_INFO: `/v3/projects/259a2dddcef84d0cabdc98c4e3678367/users/7bd71266e94347918bb4bc520d65cd41/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:08.525482 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80fe67f-c7a1-4bf7-846f-3f9943c5b702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=259a2dddcef84d0cabdc98c4e3678367, user_id=7bd71266e94347918bb4bc520d65cd41, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:08.528194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80fe67f-c7a1-4bf7-846f-3f9943c5b702 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:08.625783 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1427/2849] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:46:08 2022] PUT /identity/v3/projects/259a2dddcef84d0cabdc98c4e3678367/users/7bd71266e94347918bb4bc520d65cd41/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:46:08.629690 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a625e1-1501-4606-9e54-9be80b8f7f5b admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:08.636134 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:46:08.639862 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a625e1-1501-4606-9e54-9be80b8f7f5b admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:08.697755 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3da097-7d65-4250-aa71-cabecea11fd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:08.698453 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3da097-7d65-4250-aa71-cabecea11fd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:08.700750 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3da097-7d65-4250-aa71-cabecea11fd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:08.700750 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3da097-7d65-4250-aa71-cabecea11fd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:08.700750 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3da097-7d65-4250-aa71-cabecea11fd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:08.701487 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3da097-7d65-4250-aa71-cabecea11fd8 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:08.712848 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1423/2850] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:46:08 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:46:08.717081 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80fe67f-c7a1-4bf7-846f-3f9943c5b702 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:08.728089 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80fe67f-c7a1-4bf7-846f-3f9943c5b702 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:08.732764 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16df7a-1a5a-4de2-8854-397717b9d7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:08.733644 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16df7a-1a5a-4de2-8854-397717b9d7cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:08.733931 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16df7a-1a5a-4de2-8854-397717b9d7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:08.734209 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16df7a-1a5a-4de2-8854-397717b9d7cc admin admin] PATH_INFO: `/v3/projects/259a2dddcef84d0cabdc98c4e3678367/users/7bd71266e94347918bb4bc520d65cd41/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:08.736209 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d16df7a-1a5a-4de2-8854-397717b9d7cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=259a2dddcef84d0cabdc98c4e3678367, user_id=7bd71266e94347918bb4bc520d65cd41, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:08.738890 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d16df7a-1a5a-4de2-8854-397717b9d7cc admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:08.749034 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1428/2851] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:46:08 2022] PUT /identity/v3/projects/259a2dddcef84d0cabdc98c4e3678367/users/7bd71266e94347918bb4bc520d65cd41/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:46:08.760526 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140548a-012b-43c6-9e27-e0a76c684f74 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:08.760819 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140548a-012b-43c6-9e27-e0a76c684f74 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:08.761100 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3140548a-012b-43c6-9e27-e0a76c684f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:08.788221 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-3140548a-012b-43c6-9e27-e0a76c684f74 None None] MFA Rules not processed for user `7bd71266e94347918bb4bc520d65cd41`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:46:08.789446 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-3140548a-012b-43c6-9e27-e0a76c684f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:08.857613 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1424/2852] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:46:08 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:46:08.869426 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16df7a-1a5a-4de2-8854-397717b9d7cc admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:08.871120 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-8d16df7a-1a5a-4de2-8854-397717b9d7cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:08.928305 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16df7a-1a5a-4de2-8854-397717b9d7cc admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:08.931996 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0dd0fc-480c-4bc9-826d-6ac0e17486e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:08.932639 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0dd0fc-480c-4bc9-826d-6ac0e17486e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:08.932923 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0dd0fc-480c-4bc9-826d-6ac0e17486e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:08.933201 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0dd0fc-480c-4bc9-826d-6ac0e17486e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:08.942275 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0dd0fc-480c-4bc9-826d-6ac0e17486e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:08.943865 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0dd0fc-480c-4bc9-826d-6ac0e17486e2 service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:08.953443 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1429/2853] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:46:08 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:46:09.083321 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140548a-012b-43c6-9e27-e0a76c684f74 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:09.085972 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-3140548a-012b-43c6-9e27-e0a76c684f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:09.097927 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3140548a-012b-43c6-9e27-e0a76c684f74 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:09.102477 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba343d6-b4c4-43ce-86ca-ca87f415ab22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:09.103169 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba343d6-b4c4-43ce-86ca-ca87f415ab22 service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:09.103483 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba343d6-b4c4-43ce-86ca-ca87f415ab22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:09.103758 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba343d6-b4c4-43ce-86ca-ca87f415ab22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:09.160635 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba343d6-b4c4-43ce-86ca-ca87f415ab22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:09.162725 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba343d6-b4c4-43ce-86ca-ca87f415ab22 service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:09.172027 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1425/2854] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:46:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:46:13.268924 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0dd0fc-480c-4bc9-826d-6ac0e17486e2 service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:13.330061 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0dd0fc-480c-4bc9-826d-6ac0e17486e2 service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:13.335172 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e85a8-6882-42c4-a90b-c05aa3d60d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:13.336068 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e85a8-6882-42c4-a90b-c05aa3d60d29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:13.336383 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e85a8-6882-42c4-a90b-c05aa3d60d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:13.336729 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8e85a8-6882-42c4-a90b-c05aa3d60d29 admin admin] PATH_INFO: `/v3/users/45e2a0b6de6d48f3944bcf6162cc531f` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:13.337909 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8e85a8-6882-42c4-a90b-c05aa3d60d29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45e2a0b6de6d48f3944bcf6162cc531f)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:13.339934 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8e85a8-6882-42c4-a90b-c05aa3d60d29 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:13.361931 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:46:13.594252 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ec8e85a8-6882-42c4-a90b-c05aa3d60d29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45e2a0b6de6d48f3944bcf6162cc531f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:13.595256 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ec8e85a8-6882-42c4-a90b-c05aa3d60d29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45e2a0b6de6d48f3944bcf6162cc531f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:13.602389 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ec8e85a8-6882-42c4-a90b-c05aa3d60d29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45e2a0b6de6d48f3944bcf6162cc531f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:13.603014 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ec8e85a8-6882-42c4-a90b-c05aa3d60d29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45e2a0b6de6d48f3944bcf6162cc531f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:13.686512 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ec8e85a8-6882-42c4-a90b-c05aa3d60d29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45e2a0b6de6d48f3944bcf6162cc531f'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:13.704954 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1430/2855] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:46:13 2022] DELETE /identity/v3/users/45e2a0b6de6d48f3944bcf6162cc531f => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:46:13.709771 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b7108-726a-4d92-99cb-896089ac0af5 None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:13.710248 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b7108-726a-4d92-99cb-896089ac0af5 None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:13.710349 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2b7108-726a-4d92-99cb-896089ac0af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:13.726993 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-0d2b7108-726a-4d92-99cb-896089ac0af5 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:46:13.728258 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-0d2b7108-726a-4d92-99cb-896089ac0af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:13.748294 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=100 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:46:13.787051 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1426/2856] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:46:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:46:13.797640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e85a8-6882-42c4-a90b-c05aa3d60d29 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:13.799208 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ec8e85a8-6882-42c4-a90b-c05aa3d60d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:13.858396 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8e85a8-6882-42c4-a90b-c05aa3d60d29 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:13.861857 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9462a576-3106-4555-900f-d40fb30cedac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:13.862506 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9462a576-3106-4555-900f-d40fb30cedac service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:13.862789 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9462a576-3106-4555-900f-d40fb30cedac service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:13.863069 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9462a576-3106-4555-900f-d40fb30cedac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:13.872699 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9462a576-3106-4555-900f-d40fb30cedac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:13.874142 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9462a576-3106-4555-900f-d40fb30cedac service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:13.884817 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1431/2857] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:46:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:46:14.296431 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b7108-726a-4d92-99cb-896089ac0af5 None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:14.299636 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-0d2b7108-726a-4d92-99cb-896089ac0af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:14.322748 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d2b7108-726a-4d92-99cb-896089ac0af5 None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:14.327985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2bcf2f-3988-4930-af8a-dc5752e39a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:14.328876 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bcf2f-3988-4930-af8a-dc5752e39a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:14.329159 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bcf2f-3988-4930-af8a-dc5752e39a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:14.329435 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2bcf2f-3988-4930-af8a-dc5752e39a49 admin admin] PATH_INFO: `/v3/projects/e152d782df4f45fd8e2002887efcb694` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:14.331187 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2bcf2f-3988-4930-af8a-dc5752e39a49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e152d782df4f45fd8e2002887efcb694)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:14.333241 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2bcf2f-3988-4930-af8a-dc5752e39a49 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:14.483606 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2f2bcf2f-3988-4930-af8a-dc5752e39a49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e152d782df4f45fd8e2002887efcb694', 'request_id': 'req-2f2bcf2f-3988-4930-af8a-dc5752e39a49'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:14.493756 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2f2bcf2f-3988-4930-af8a-dc5752e39a49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e152d782df4f45fd8e2002887efcb694', 'request_id': 'req-2f2bcf2f-3988-4930-af8a-dc5752e39a49'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:14.504118 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2f2bcf2f-3988-4930-af8a-dc5752e39a49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e152d782df4f45fd8e2002887efcb694', 'request_id': 'req-2f2bcf2f-3988-4930-af8a-dc5752e39a49'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:14.595474 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2f2bcf2f-3988-4930-af8a-dc5752e39a49 admin admin] The token cache is being invalidate because project e152d782df4f45fd8e2002887efcb694 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:46:14.595760 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2f2bcf2f-3988-4930-af8a-dc5752e39a49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:14.596376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-2f2bcf2f-3988-4930-af8a-dc5752e39a49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:14.597929 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1427/2858] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:46:14 2022] DELETE /identity/v3/projects/e152d782df4f45fd8e2002887efcb694 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:46:14.602530 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9462a576-3106-4555-900f-d40fb30cedac service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:14.612411 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9462a576-3106-4555-900f-d40fb30cedac service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:14.668276 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea39e4c-ffa8-4e11-8e75-62fa2cb64110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:14.669146 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea39e4c-ffa8-4e11-8e75-62fa2cb64110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:14.669430 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea39e4c-ffa8-4e11-8e75-62fa2cb64110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:14.669706 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea39e4c-ffa8-4e11-8e75-62fa2cb64110 admin admin] PATH_INFO: `/v3/users/9fa423226f964615a648b0ee016efffa` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:14.670887 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea39e4c-ffa8-4e11-8e75-62fa2cb64110 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fa423226f964615a648b0ee016efffa)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:14.672851 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea39e4c-ffa8-4e11-8e75-62fa2cb64110 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:14.855282 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:46:14.857664 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fea39e4c-ffa8-4e11-8e75-62fa2cb64110 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fa423226f964615a648b0ee016efffa'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:14.858313 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fea39e4c-ffa8-4e11-8e75-62fa2cb64110 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fa423226f964615a648b0ee016efffa'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:14.865584 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fea39e4c-ffa8-4e11-8e75-62fa2cb64110 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fa423226f964615a648b0ee016efffa'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:14.866217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fea39e4c-ffa8-4e11-8e75-62fa2cb64110 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fa423226f964615a648b0ee016efffa'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:14.977974 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-fea39e4c-ffa8-4e11-8e75-62fa2cb64110 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fa423226f964615a648b0ee016efffa'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:14.996437 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1432/2859] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:46:14 2022] DELETE /identity/v3/users/9fa423226f964615a648b0ee016efffa => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:46:15.422319 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2bcf2f-3988-4930-af8a-dc5752e39a49 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:15.423809 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-2f2bcf2f-3988-4930-af8a-dc5752e39a49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:15.439370 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2bcf2f-3988-4930-af8a-dc5752e39a49 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:15.498408 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1e3d5-62b3-4691-bd42-dd78e2836219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:15.499321 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1e3d5-62b3-4691-bd42-dd78e2836219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:15.499612 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1e3d5-62b3-4691-bd42-dd78e2836219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:15.499894 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1e3d5-62b3-4691-bd42-dd78e2836219 admin admin] PATH_INFO: `/v3/projects/1f800e23481c4709816443e271322eec` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:15.501102 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1e3d5-62b3-4691-bd42-dd78e2836219 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f800e23481c4709816443e271322eec)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:15.503113 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1e3d5-62b3-4691-bd42-dd78e2836219 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:15.658286 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bca1e3d5-62b3-4691-bd42-dd78e2836219 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f800e23481c4709816443e271322eec', 'request_id': 'req-bca1e3d5-62b3-4691-bd42-dd78e2836219'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:15.659364 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:46:15.666048 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bca1e3d5-62b3-4691-bd42-dd78e2836219 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f800e23481c4709816443e271322eec', 'request_id': 'req-bca1e3d5-62b3-4691-bd42-dd78e2836219'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:15.671376 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bca1e3d5-62b3-4691-bd42-dd78e2836219 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f800e23481c4709816443e271322eec', 'request_id': 'req-bca1e3d5-62b3-4691-bd42-dd78e2836219'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:15.790149 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bca1e3d5-62b3-4691-bd42-dd78e2836219 admin admin] The token cache is being invalidate because project 1f800e23481c4709816443e271322eec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:46:15.790441 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bca1e3d5-62b3-4691-bd42-dd78e2836219 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:15.791104 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-bca1e3d5-62b3-4691-bd42-dd78e2836219 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:15.792704 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1428/2860] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:46:15 2022] DELETE /identity/v3/projects/1f800e23481c4709816443e271322eec => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 30 10:46:15.796788 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea39e4c-ffa8-4e11-8e75-62fa2cb64110 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:15.798201 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-fea39e4c-ffa8-4e11-8e75-62fa2cb64110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:15.812118 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea39e4c-ffa8-4e11-8e75-62fa2cb64110 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:15.870217 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2767c50f-1fcf-4133-bc4d-e38ce9e4b046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:15.871096 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767c50f-1fcf-4133-bc4d-e38ce9e4b046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:15.871413 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767c50f-1fcf-4133-bc4d-e38ce9e4b046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:15.871691 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2767c50f-1fcf-4133-bc4d-e38ce9e4b046 admin admin] PATH_INFO: `/v3/users/7bd71266e94347918bb4bc520d65cd41` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:15.872930 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767c50f-1fcf-4133-bc4d-e38ce9e4b046 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bd71266e94347918bb4bc520d65cd41)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:15.874850 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2767c50f-1fcf-4133-bc4d-e38ce9e4b046 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:16.056699 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2767c50f-1fcf-4133-bc4d-e38ce9e4b046 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bd71266e94347918bb4bc520d65cd41'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:16.057262 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2767c50f-1fcf-4133-bc4d-e38ce9e4b046 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bd71266e94347918bb4bc520d65cd41'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:16.064191 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2767c50f-1fcf-4133-bc4d-e38ce9e4b046 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bd71266e94347918bb4bc520d65cd41'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:16.065973 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2767c50f-1fcf-4133-bc4d-e38ce9e4b046 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bd71266e94347918bb4bc520d65cd41'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:16.138130 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-2767c50f-1fcf-4133-bc4d-e38ce9e4b046 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bd71266e94347918bb4bc520d65cd41'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:16.156759 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1433/2861] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:46:15 2022] DELETE /identity/v3/users/7bd71266e94347918bb4bc520d65cd41 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:46:16.651061 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1e3d5-62b3-4691-bd42-dd78e2836219 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:16.652616 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-bca1e3d5-62b3-4691-bd42-dd78e2836219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:16.670230 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1e3d5-62b3-4691-bd42-dd78e2836219 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:16.724198 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118a2f02-96ed-4584-b32d-6c883ad759e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:16.725628 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a2f02-96ed-4584-b32d-6c883ad759e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:16.726062 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a2f02-96ed-4584-b32d-6c883ad759e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:16.726566 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118a2f02-96ed-4584-b32d-6c883ad759e6 admin admin] PATH_INFO: `/v3/projects/259a2dddcef84d0cabdc98c4e3678367` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:16.728072 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a2f02-96ed-4584-b32d-6c883ad759e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=259a2dddcef84d0cabdc98c4e3678367)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:16.730690 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118a2f02-96ed-4584-b32d-6c883ad759e6 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:16.836670 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-118a2f02-96ed-4584-b32d-6c883ad759e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '259a2dddcef84d0cabdc98c4e3678367', 'request_id': 'req-118a2f02-96ed-4584-b32d-6c883ad759e6'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:16.842667 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-118a2f02-96ed-4584-b32d-6c883ad759e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '259a2dddcef84d0cabdc98c4e3678367', 'request_id': 'req-118a2f02-96ed-4584-b32d-6c883ad759e6'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:16.848074 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-118a2f02-96ed-4584-b32d-6c883ad759e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '259a2dddcef84d0cabdc98c4e3678367', 'request_id': 'req-118a2f02-96ed-4584-b32d-6c883ad759e6'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:17.012734 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-118a2f02-96ed-4584-b32d-6c883ad759e6 admin admin] The token cache is being invalidate because project 259a2dddcef84d0cabdc98c4e3678367 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:46:17.013067 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-118a2f02-96ed-4584-b32d-6c883ad759e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:17.013646 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-118a2f02-96ed-4584-b32d-6c883ad759e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:17.015172 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1429/2862] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:46:16 2022] DELETE /identity/v3/projects/259a2dddcef84d0cabdc98c4e3678367 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:46:17.021266 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66672f47-8008-456a-8306-15e948572319 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:17.022366 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66672f47-8008-456a-8306-15e948572319 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:17.022366 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66672f47-8008-456a-8306-15e948572319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:17.050296 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-66672f47-8008-456a-8306-15e948572319 None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:46:17.051525 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-66672f47-8008-456a-8306-15e948572319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:17.077684 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:46:17.112112 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1434/2863] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:46:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:46:17.126510 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a7d64-d1a6-4d5c-9b96-b07cd040aabd None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:17.126797 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a7d64-d1a6-4d5c-9b96-b07cd040aabd None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:17.127072 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a7d64-d1a6-4d5c-9b96-b07cd040aabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:17.145099 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-a09a7d64-d1a6-4d5c-9b96-b07cd040aabd None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:46:17.146576 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-a09a7d64-d1a6-4d5c-9b96-b07cd040aabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:17.154733 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1430/2864] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:46:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:46:17.159011 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66672f47-8008-456a-8306-15e948572319 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:17.169502 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66672f47-8008-456a-8306-15e948572319 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:17.174542 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516dd9a-597a-4080-959f-517041b2a592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:17.175191 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516dd9a-597a-4080-959f-517041b2a592 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:17.175510 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516dd9a-597a-4080-959f-517041b2a592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:17.175788 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2516dd9a-597a-4080-959f-517041b2a592 admin admin] PATH_INFO: `/v3/domains` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:17.176545 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2516dd9a-597a-4080-959f-517041b2a592 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80363) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 30 10:46:17.177432 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2516dd9a-597a-4080-959f-517041b2a592 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:17.179048 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2516dd9a-597a-4080-959f-517041b2a592 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:17.190591 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1435/2865] 10.222.0.47 () {58 vars in 1152 bytes} [Thu Jun 30 10:46:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 30 10:46:17.194895 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a7d64-d1a6-4d5c-9b96-b07cd040aabd None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:17.204056 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a7d64-d1a6-4d5c-9b96-b07cd040aabd None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:17.208779 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b53cd5-c1ca-4adb-a529-f9a61b5d5676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:17.209432 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b53cd5-c1ca-4adb-a529-f9a61b5d5676 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:17.209716 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b53cd5-c1ca-4adb-a529-f9a61b5d5676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:17.209991 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b53cd5-c1ca-4adb-a529-f9a61b5d5676 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:17.210695 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b53cd5-c1ca-4adb-a529-f9a61b5d5676 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:17.212637 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b53cd5-c1ca-4adb-a529-f9a61b5d5676 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:17.313589 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1431/2866] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:46:17 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:46:17.317511 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516dd9a-597a-4080-959f-517041b2a592 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:17.327554 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2516dd9a-597a-4080-959f-517041b2a592 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:17.383971 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ae3e6-8707-49b3-8d72-af3423ce8210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:17.384640 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ae3e6-8707-49b3-8d72-af3423ce8210 admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:17.384922 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ae3e6-8707-49b3-8d72-af3423ce8210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:17.385211 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ae3e6-8707-49b3-8d72-af3423ce8210 admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:17.386052 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ae3e6-8707-49b3-8d72-af3423ce8210 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:17.388025 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641ae3e6-8707-49b3-8d72-af3423ce8210 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:17.482761 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1436/2867] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:46:17 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:46:17.486742 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b53cd5-c1ca-4adb-a529-f9a61b5d5676 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:17.497679 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b53cd5-c1ca-4adb-a529-f9a61b5d5676 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:17.502425 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e1312-9e16-4949-8a80-8f2c27495199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:17.503092 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e1312-9e16-4949-8a80-8f2c27495199 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:17.503411 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e1312-9e16-4949-8a80-8f2c27495199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:17.503690 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356e1312-9e16-4949-8a80-8f2c27495199 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:17.504429 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e1312-9e16-4949-8a80-8f2c27495199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:17.506037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356e1312-9e16-4949-8a80-8f2c27495199 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:17.516746 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1432/2868] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:46:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:46:17.521506 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ae3e6-8707-49b3-8d72-af3423ce8210 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:17.531472 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ae3e6-8707-49b3-8d72-af3423ce8210 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:17.536672 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e31f23e-51a2-4bf3-812b-2c5be59c4a63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:17.537485 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31f23e-51a2-4bf3-812b-2c5be59c4a63 admin admin] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:17.537839 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31f23e-51a2-4bf3-812b-2c5be59c4a63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:17.538121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e31f23e-51a2-4bf3-812b-2c5be59c4a63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:17.538847 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e31f23e-51a2-4bf3-812b-2c5be59c4a63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:17.540746 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e31f23e-51a2-4bf3-812b-2c5be59c4a63 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:17.550423 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1437/2869] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:46:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:46:17.554341 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e1312-9e16-4949-8a80-8f2c27495199 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:17.564124 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356e1312-9e16-4949-8a80-8f2c27495199 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:17.569047 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24348e1b-a812-4ea3-9db1-37b173f5e3c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:17.569913 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24348e1b-a812-4ea3-9db1-37b173f5e3c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:17.570195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24348e1b-a812-4ea3-9db1-37b173f5e3c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:17.570471 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24348e1b-a812-4ea3-9db1-37b173f5e3c3 admin admin] PATH_INFO: `/v3/projects/9cce0cc077724252bee1b8472db5ebeb/users/59eb34e3c63545bcaa66052c0b819d8a/roles/31034f95ef88402296c753c2c259f717` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:17.585744 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24348e1b-a812-4ea3-9db1-37b173f5e3c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cce0cc077724252bee1b8472db5ebeb, user_id=59eb34e3c63545bcaa66052c0b819d8a, role_id=31034f95ef88402296c753c2c259f717)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:17.588486 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24348e1b-a812-4ea3-9db1-37b173f5e3c3 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:17.694819 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1433/2870] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:46:17 2022] PUT /identity/v3/projects/9cce0cc077724252bee1b8472db5ebeb/users/59eb34e3c63545bcaa66052c0b819d8a/roles/31034f95ef88402296c753c2c259f717 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:46:17.706823 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f063f-9d5b-4cd4-b6c7-af9ff8798905 None None] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:17.709960 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f063f-9d5b-4cd4-b6c7-af9ff8798905 None None] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:17.709960 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f063f-9d5b-4cd4-b6c7-af9ff8798905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:17.734569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.auth.core [None req-a41f063f-9d5b-4cd4-b6c7-af9ff8798905 None None] MFA Rules not processed for user `59eb34e3c63545bcaa66052c0b819d8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80363) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:46:17.735852 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a41f063f-9d5b-4cd4-b6c7-af9ff8798905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:17.799020 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1438/2871] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:46:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:46:17.803941 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24348e1b-a812-4ea3-9db1-37b173f5e3c3 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:17.809479 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:46:17.865878 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24348e1b-a812-4ea3-9db1-37b173f5e3c3 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:17.870681 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8147f-c1a0-46bd-b6dd-f8a113bf5065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:17.871733 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8147f-c1a0-46bd-b6dd-f8a113bf5065 admin admin] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:17.872030 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8147f-c1a0-46bd-b6dd-f8a113bf5065 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:17.872317 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a8147f-c1a0-46bd-b6dd-f8a113bf5065 admin admin] PATH_INFO: `/v3/projects` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:17.873050 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8147f-c1a0-46bd-b6dd-f8a113bf5065 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:17.875360 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a8147f-c1a0-46bd-b6dd-f8a113bf5065 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:17.970289 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1434/2872] 10.222.0.47 () {60 vars in 1152 bytes} [Thu Jun 30 10:46:17 2022] POST /identity/v3/projects => generated 385 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 30 10:46:17.974231 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f063f-9d5b-4cd4-b6c7-af9ff8798905 None None] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:17.985308 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f063f-9d5b-4cd4-b6c7-af9ff8798905 None None] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:18.009631 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:46:18.042524 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391bd4c7-f3c5-4209-a8f1-d2e7f6b0450d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:18.043200 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bd4c7-f3c5-4209-a8f1-d2e7f6b0450d admin admin] REQUEST_METHOD: `POST` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:18.043516 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bd4c7-f3c5-4209-a8f1-d2e7f6b0450d admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:18.043795 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391bd4c7-f3c5-4209-a8f1-d2e7f6b0450d admin admin] PATH_INFO: `/v3/users` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:18.044496 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391bd4c7-f3c5-4209-a8f1-d2e7f6b0450d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:18.046428 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391bd4c7-f3c5-4209-a8f1-d2e7f6b0450d admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:18.144772 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1439/2873] 10.222.0.47 () {60 vars in 1143 bytes} [Thu Jun 30 10:46:17 2022] POST /identity/v3/users => generated 299 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 30 10:46:18.149442 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8147f-c1a0-46bd-b6dd-f8a113bf5065 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:18.159809 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a8147f-c1a0-46bd-b6dd-f8a113bf5065 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:18.164985 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f1ec64-6409-4a29-8edc-9759de91fca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:18.173628 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1ec64-6409-4a29-8edc-9759de91fca0 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:18.173628 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1ec64-6409-4a29-8edc-9759de91fca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:18.173628 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1ec64-6409-4a29-8edc-9759de91fca0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:18.173628 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f1ec64-6409-4a29-8edc-9759de91fca0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:18.173628 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f1ec64-6409-4a29-8edc-9759de91fca0 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:18.179508 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1435/2874] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:46:18.183643 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391bd4c7-f3c5-4209-a8f1-d2e7f6b0450d admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:18.194464 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391bd4c7-f3c5-4209-a8f1-d2e7f6b0450d admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:18.199093 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec65268-893c-4f91-a6ca-50e85d478074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:18.200002 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec65268-893c-4f91-a6ca-50e85d478074 admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:18.200290 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec65268-893c-4f91-a6ca-50e85d478074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:18.200569 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec65268-893c-4f91-a6ca-50e85d478074 admin admin] PATH_INFO: `/v3/projects/3861bc0234cd4693a7551cfee62cbf45/users/3dcf77c0f2c1465fa378c5c88b46e92b/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:18.216701 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec65268-893c-4f91-a6ca-50e85d478074 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3861bc0234cd4693a7551cfee62cbf45, user_id=3dcf77c0f2c1465fa378c5c88b46e92b, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:18.219562 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec65268-893c-4f91-a6ca-50e85d478074 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:18.340242 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1440/2875] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:46:18 2022] PUT /identity/v3/projects/3861bc0234cd4693a7551cfee62cbf45/users/3dcf77c0f2c1465fa378c5c88b46e92b/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:46:18.344298 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f1ec64-6409-4a29-8edc-9759de91fca0 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:18.354861 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f1ec64-6409-4a29-8edc-9759de91fca0 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:18.410845 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d44cab-ad65-4562-94c8-13368738b569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:18.411512 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d44cab-ad65-4562-94c8-13368738b569 admin admin] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:18.411800 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d44cab-ad65-4562-94c8-13368738b569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:18.412079 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d44cab-ad65-4562-94c8-13368738b569 admin admin] PATH_INFO: `/v3/roles` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:18.412829 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d44cab-ad65-4562-94c8-13368738b569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:18.414425 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d44cab-ad65-4562-94c8-13368738b569 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:18.424206 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1436/2876] 10.222.0.47 () {58 vars in 1121 bytes} [Thu Jun 30 10:46:18 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 30 10:46:18.429036 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec65268-893c-4f91-a6ca-50e85d478074 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:18.439419 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec65268-893c-4f91-a6ca-50e85d478074 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:18.444957 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca07186-1c73-4870-98bf-1a5a04cad89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:18.445843 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca07186-1c73-4870-98bf-1a5a04cad89b admin admin] REQUEST_METHOD: `PUT` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:18.446130 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca07186-1c73-4870-98bf-1a5a04cad89b admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:18.446409 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca07186-1c73-4870-98bf-1a5a04cad89b admin admin] PATH_INFO: `/v3/projects/3861bc0234cd4693a7551cfee62cbf45/users/3dcf77c0f2c1465fa378c5c88b46e92b/roles/e63bda042e104a4a93fba15ffcfe1183` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:18.448405 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca07186-1c73-4870-98bf-1a5a04cad89b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3861bc0234cd4693a7551cfee62cbf45, user_id=3dcf77c0f2c1465fa378c5c88b46e92b, role_id=e63bda042e104a4a93fba15ffcfe1183)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:18.451113 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca07186-1c73-4870-98bf-1a5a04cad89b admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:18.461362 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1441/2877] 10.222.0.47 () {60 vars in 1482 bytes} [Thu Jun 30 10:46:18 2022] PUT /identity/v3/projects/3861bc0234cd4693a7551cfee62cbf45/users/3dcf77c0f2c1465fa378c5c88b46e92b/roles/e63bda042e104a4a93fba15ffcfe1183 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:46:18.471358 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172840e5-5b23-44ce-a362-36e8f121165a None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:18.471651 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172840e5-5b23-44ce-a362-36e8f121165a None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:18.471928 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172840e5-5b23-44ce-a362-36e8f121165a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:18.498674 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-172840e5-5b23-44ce-a362-36e8f121165a None None] MFA Rules not processed for user `3dcf77c0f2c1465fa378c5c88b46e92b`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:46:18.499951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-172840e5-5b23-44ce-a362-36e8f121165a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:18.563339 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:46:18.567071 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1437/2878] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:46:18 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 30 10:46:18.578489 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca07186-1c73-4870-98bf-1a5a04cad89b admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:18.580289 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-2ca07186-1c73-4870-98bf-1a5a04cad89b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:18.657622 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca07186-1c73-4870-98bf-1a5a04cad89b admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:18.661843 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf50dbd-db85-41c9-aa28-c7509fee12ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:18.662564 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf50dbd-db85-41c9-aa28-c7509fee12ec service cinder] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:18.662855 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf50dbd-db85-41c9-aa28-c7509fee12ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:18.663127 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf50dbd-db85-41c9-aa28-c7509fee12ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:18.718374 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf50dbd-db85-41c9-aa28-c7509fee12ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:18.719862 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf50dbd-db85-41c9-aa28-c7509fee12ec service cinder] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:18.733847 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1442/2879] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:46:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 30 10:46:24.046785 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172840e5-5b23-44ce-a362-36e8f121165a None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:24.048195 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-172840e5-5b23-44ce-a362-36e8f121165a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:24.060680 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172840e5-5b23-44ce-a362-36e8f121165a None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:24.064252 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a1b339-ff29-426f-845c-fb420f5d8358 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07ab12e5d1d94a009c4d414e5c43ac92', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:24.064894 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1b339-ff29-426f-845c-fb420f5d8358 service cinder] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:24.065182 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1b339-ff29-426f-845c-fb420f5d8358 service cinder] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:24.065463 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a1b339-ff29-426f-845c-fb420f5d8358 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:24.074377 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1b339-ff29-426f-845c-fb420f5d8358 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:24.075869 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a1b339-ff29-426f-845c-fb420f5d8358 service cinder] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:24.085891 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1438/2880] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:46:24 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:46:28.729560 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:INSERT=5 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:46:30.309201 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf50dbd-db85-41c9-aa28-c7509fee12ec service cinder] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:30.371409 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf50dbd-db85-41c9-aa28-c7509fee12ec service cinder] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:30.375882 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd0d3e-8d2c-4282-8f6d-ee33cb471408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:30.376753 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd0d3e-8d2c-4282-8f6d-ee33cb471408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:30.377038 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd0d3e-8d2c-4282-8f6d-ee33cb471408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:30.377314 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fd0d3e-8d2c-4282-8f6d-ee33cb471408 admin admin] PATH_INFO: `/v3/users/59eb34e3c63545bcaa66052c0b819d8a` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:30.378479 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fd0d3e-8d2c-4282-8f6d-ee33cb471408 admin admin] RBAC: Authorizing `identity:delete_user(user_id=59eb34e3c63545bcaa66052c0b819d8a)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:30.381536 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fd0d3e-8d2c-4282-8f6d-ee33cb471408 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:30.620472 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a3fd0d3e-8d2c-4282-8f6d-ee33cb471408 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59eb34e3c63545bcaa66052c0b819d8a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:30.621045 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a3fd0d3e-8d2c-4282-8f6d-ee33cb471408 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59eb34e3c63545bcaa66052c0b819d8a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:30.627919 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a3fd0d3e-8d2c-4282-8f6d-ee33cb471408 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59eb34e3c63545bcaa66052c0b819d8a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:30.628537 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a3fd0d3e-8d2c-4282-8f6d-ee33cb471408 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59eb34e3c63545bcaa66052c0b819d8a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:30.697645 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-a3fd0d3e-8d2c-4282-8f6d-ee33cb471408 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59eb34e3c63545bcaa66052c0b819d8a'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:30.715296 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1443/2881] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:46:30 2022] DELETE /identity/v3/users/59eb34e3c63545bcaa66052c0b819d8a => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:46:30.719808 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ce974-4447-4b91-b9af-7a7fafacec9b None None] REQUEST_METHOD: `POST` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:30.720272 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ce974-4447-4b91-b9af-7a7fafacec9b None None] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:30.720415 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036ce974-4447-4b91-b9af-7a7fafacec9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:30.739575 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.auth.core [None req-036ce974-4447-4b91-b9af-7a7fafacec9b None None] MFA Rules not processed for user `67e761d82cde42509f040f82a44c86c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80364) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 30 10:46:30.740859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-036ce974-4447-4b91-b9af-7a7fafacec9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:30.800025 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1439/2882] 10.222.0.47 () {58 vars in 957 bytes} [Thu Jun 30 10:46:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 30 10:46:30.810814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd0d3e-8d2c-4282-8f6d-ee33cb471408 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:30.812215 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-a3fd0d3e-8d2c-4282-8f6d-ee33cb471408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:30.868416 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fd0d3e-8d2c-4282-8f6d-ee33cb471408 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:30.872110 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ca2a9-3287-4011-8808-8d98e62f64bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:30.872688 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ca2a9-3287-4011-8808-8d98e62f64bd service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:30.872976 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ca2a9-3287-4011-8808-8d98e62f64bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:30.873249 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73ca2a9-3287-4011-8808-8d98e62f64bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:30.882121 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ca2a9-3287-4011-8808-8d98e62f64bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:30.883593 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73ca2a9-3287-4011-8808-8d98e62f64bd service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:30.892796 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1444/2883] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:46:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:46:31.316571 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ce974-4447-4b91-b9af-7a7fafacec9b None None] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:31.318951 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-036ce974-4447-4b91-b9af-7a7fafacec9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:31.334048 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036ce974-4447-4b91-b9af-7a7fafacec9b None None] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:31.339138 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af43170-3269-472a-ace4-cecd8911eda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:31.340037 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af43170-3269-472a-ace4-cecd8911eda1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:31.340350 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af43170-3269-472a-ace4-cecd8911eda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:31.340629 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af43170-3269-472a-ace4-cecd8911eda1 admin admin] PATH_INFO: `/v3/projects/9cce0cc077724252bee1b8472db5ebeb` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:31.341827 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af43170-3269-472a-ace4-cecd8911eda1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cce0cc077724252bee1b8472db5ebeb)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:31.343915 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af43170-3269-472a-ace4-cecd8911eda1 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:31.458870 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-0af43170-3269-472a-ace4-cecd8911eda1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cce0cc077724252bee1b8472db5ebeb', 'request_id': 'req-0af43170-3269-472a-ace4-cecd8911eda1'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:31.465035 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-0af43170-3269-472a-ace4-cecd8911eda1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cce0cc077724252bee1b8472db5ebeb', 'request_id': 'req-0af43170-3269-472a-ace4-cecd8911eda1'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:31.470564 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-0af43170-3269-472a-ace4-cecd8911eda1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cce0cc077724252bee1b8472db5ebeb', 'request_id': 'req-0af43170-3269-472a-ace4-cecd8911eda1'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:31.560297 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-0af43170-3269-472a-ace4-cecd8911eda1 admin admin] The token cache is being invalidate because project 9cce0cc077724252bee1b8472db5ebeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:46:31.560589 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-0af43170-3269-472a-ace4-cecd8911eda1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:31.561240 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-0af43170-3269-472a-ace4-cecd8911eda1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:31.562822 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1440/2884] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:46:31 2022] DELETE /identity/v3/projects/9cce0cc077724252bee1b8472db5ebeb => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:46:31.567355 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ca2a9-3287-4011-8808-8d98e62f64bd service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:31.577472 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73ca2a9-3287-4011-8808-8d98e62f64bd service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:31.596014 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:46:31.633363 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37504c5-1a3e-4966-8fd5-195dab74f165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:31.634248 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37504c5-1a3e-4966-8fd5-195dab74f165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:31.634536 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37504c5-1a3e-4966-8fd5-195dab74f165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:31.634820 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37504c5-1a3e-4966-8fd5-195dab74f165 admin admin] PATH_INFO: `/v3/users/3dcf77c0f2c1465fa378c5c88b46e92b` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:31.636113 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37504c5-1a3e-4966-8fd5-195dab74f165 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3dcf77c0f2c1465fa378c5c88b46e92b)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:31.638194 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37504c5-1a3e-4966-8fd5-195dab74f165 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:31.911357 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e37504c5-1a3e-4966-8fd5-195dab74f165 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3dcf77c0f2c1465fa378c5c88b46e92b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:31.911967 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e37504c5-1a3e-4966-8fd5-195dab74f165 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3dcf77c0f2c1465fa378c5c88b46e92b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:31.921692 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e37504c5-1a3e-4966-8fd5-195dab74f165 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3dcf77c0f2c1465fa378c5c88b46e92b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:31.922353 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e37504c5-1a3e-4966-8fd5-195dab74f165 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3dcf77c0f2c1465fa378c5c88b46e92b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:31.991259 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-e37504c5-1a3e-4966-8fd5-195dab74f165 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3dcf77c0f2c1465fa378c5c88b46e92b'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:32.010173 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1445/2885] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:46:31 2022] DELETE /identity/v3/users/3dcf77c0f2c1465fa378c5c88b46e92b => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:46:32.529081 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af43170-3269-472a-ace4-cecd8911eda1 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:32.530557 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-0af43170-3269-472a-ace4-cecd8911eda1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:32.544970 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af43170-3269-472a-ace4-cecd8911eda1 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:32.602866 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3f369-7dd5-4731-bf65-225ee9dc5ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:32.603752 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3f369-7dd5-4731-bf65-225ee9dc5ed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:32.604033 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3f369-7dd5-4731-bf65-225ee9dc5ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:32.604306 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c3f369-7dd5-4731-bf65-225ee9dc5ed5 admin admin] PATH_INFO: `/v3/projects/3861bc0234cd4693a7551cfee62cbf45` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:32.605482 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3f369-7dd5-4731-bf65-225ee9dc5ed5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3861bc0234cd4693a7551cfee62cbf45)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:32.607455 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c3f369-7dd5-4731-bf65-225ee9dc5ed5 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:32.619042 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:46:32.763033 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e6c3f369-7dd5-4731-bf65-225ee9dc5ed5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3861bc0234cd4693a7551cfee62cbf45', 'request_id': 'req-e6c3f369-7dd5-4731-bf65-225ee9dc5ed5'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:32.769278 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e6c3f369-7dd5-4731-bf65-225ee9dc5ed5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3861bc0234cd4693a7551cfee62cbf45', 'request_id': 'req-e6c3f369-7dd5-4731-bf65-225ee9dc5ed5'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:32.775498 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e6c3f369-7dd5-4731-bf65-225ee9dc5ed5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3861bc0234cd4693a7551cfee62cbf45', 'request_id': 'req-e6c3f369-7dd5-4731-bf65-225ee9dc5ed5'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:32.871023 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e6c3f369-7dd5-4731-bf65-225ee9dc5ed5 admin admin] The token cache is being invalidate because project 3861bc0234cd4693a7551cfee62cbf45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:46:32.871338 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e6c3f369-7dd5-4731-bf65-225ee9dc5ed5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:32.871969 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-e6c3f369-7dd5-4731-bf65-225ee9dc5ed5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:46:32.873522 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1441/2886] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:46:32 2022] DELETE /identity/v3/projects/3861bc0234cd4693a7551cfee62cbf45 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:46:42.006521 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:46:42.877106 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=19,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:46:52.456371 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37504c5-1a3e-4966-8fd5-195dab74f165 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:52.457814 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-e37504c5-1a3e-4966-8fd5-195dab74f165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:52.516125 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37504c5-1a3e-4966-8fd5-195dab74f165 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:52.564746 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c751a-423b-4689-9feb-f85d84515f6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059b05007c0144f7a50dbd9f0ef41359', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:52.565315 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c751a-423b-4689-9feb-f85d84515f6a service neutron] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:52.565596 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c751a-423b-4689-9feb-f85d84515f6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:52.565872 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414c751a-423b-4689-9feb-f85d84515f6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:52.568230 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-414c751a-423b-4689-9feb-f85d84515f6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:52.633732 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c751a-423b-4689-9feb-f85d84515f6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:52.635352 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414c751a-423b-4689-9feb-f85d84515f6a service neutron] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:52.646608 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1446/2887] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:46:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:46:56.470084 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3f369-7dd5-4731-bf65-225ee9dc5ed5 admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:46:56.471632 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-e6c3f369-7dd5-4731-bf65-225ee9dc5ed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:56.569551 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c3f369-7dd5-4731-bf65-225ee9dc5ed5 admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:46:56.574490 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cda1e2-2a08-4c34-8034-084dce2dd1aa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46c99f43a9344d7d92632990e45bb9cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:46:56.575135 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cda1e2-2a08-4c34-8034-084dce2dd1aa service placement] REQUEST_METHOD: `GET` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:46:56.575454 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cda1e2-2a08-4c34-8034-084dce2dd1aa service placement] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:46:56.575737 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cda1e2-2a08-4c34-8034-084dce2dd1aa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:46:56.577562 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-86cda1e2-2a08-4c34-8034-084dce2dd1aa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:46:56.592715 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cda1e2-2a08-4c34-8034-084dce2dd1aa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:46:56.594924 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cda1e2-2a08-4c34-8034-084dce2dd1aa service placement] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:46:56.608825 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1442/2888] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:46:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:47:00.470309 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c751a-423b-4689-9feb-f85d84515f6a service neutron] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:47:00.471849 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-414c751a-423b-4689-9feb-f85d84515f6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:47:00.487835 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414c751a-423b-4689-9feb-f85d84515f6a service neutron] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:47:00.490164 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ea653-a462-4254-a831-247136b85996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:47:00.490803 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ea653-a462-4254-a831-247136b85996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:47:00.491091 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ea653-a462-4254-a831-247136b85996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:47:00.491399 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ea653-a462-4254-a831-247136b85996 admin admin] PATH_INFO: `/v3/users/fa58de3ae9eb4dc9bd80fc41ec3e0793` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:47:00.492620 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ea653-a462-4254-a831-247136b85996 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa58de3ae9eb4dc9bd80fc41ec3e0793)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:47:00.494581 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ea653-a462-4254-a831-247136b85996 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:47:00.726816 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-396ea653-a462-4254-a831-247136b85996 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa58de3ae9eb4dc9bd80fc41ec3e0793'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:47:00.727559 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-396ea653-a462-4254-a831-247136b85996 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa58de3ae9eb4dc9bd80fc41ec3e0793'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:47:00.735394 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-396ea653-a462-4254-a831-247136b85996 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa58de3ae9eb4dc9bd80fc41ec3e0793'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:47:00.736200 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-396ea653-a462-4254-a831-247136b85996 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa58de3ae9eb4dc9bd80fc41ec3e0793'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:47:00.741265 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:47:00.835243 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-396ea653-a462-4254-a831-247136b85996 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa58de3ae9eb4dc9bd80fc41ec3e0793'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:47:00.853774 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1447/2889] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:47:00 2022] DELETE /identity/v3/users/fa58de3ae9eb4dc9bd80fc41ec3e0793 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:47:01.118026 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cda1e2-2a08-4c34-8034-084dce2dd1aa service placement] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:47:01.128674 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cda1e2-2a08-4c34-8034-084dce2dd1aa service placement] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:47:01.187435 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb9bb1-7066-4cab-a05a-0c6de942248f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:47:01.188304 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb9bb1-7066-4cab-a05a-0c6de942248f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:47:01.188615 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb9bb1-7066-4cab-a05a-0c6de942248f admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:47:01.188895 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb9bb1-7066-4cab-a05a-0c6de942248f admin admin] PATH_INFO: `/v3/projects/158457f258274015ba394ce8cfd86a9a` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:47:01.190114 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb9bb1-7066-4cab-a05a-0c6de942248f admin admin] RBAC: Authorizing `identity:delete_project(project_id=158457f258274015ba394ce8cfd86a9a)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:47:01.192095 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb9bb1-7066-4cab-a05a-0c6de942248f admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:47:01.337409 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9bbb9bb1-7066-4cab-a05a-0c6de942248f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '158457f258274015ba394ce8cfd86a9a', 'request_id': 'req-9bbb9bb1-7066-4cab-a05a-0c6de942248f'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:47:01.345096 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9bbb9bb1-7066-4cab-a05a-0c6de942248f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '158457f258274015ba394ce8cfd86a9a', 'request_id': 'req-9bbb9bb1-7066-4cab-a05a-0c6de942248f'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:47:01.351028 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9bbb9bb1-7066-4cab-a05a-0c6de942248f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '158457f258274015ba394ce8cfd86a9a', 'request_id': 'req-9bbb9bb1-7066-4cab-a05a-0c6de942248f'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:47:01.462674 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:47:01.466550 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9bbb9bb1-7066-4cab-a05a-0c6de942248f admin admin] The token cache is being invalidate because project 158457f258274015ba394ce8cfd86a9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:47:01.469971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9bbb9bb1-7066-4cab-a05a-0c6de942248f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:47:01.469971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9bbb9bb1-7066-4cab-a05a-0c6de942248f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:47:01.469971 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1443/2890] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:47:01 2022] DELETE /identity/v3/projects/158457f258274015ba394ce8cfd86a9a => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:47:01.473955 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ea653-a462-4254-a831-247136b85996 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:47:01.476033 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-396ea653-a462-4254-a831-247136b85996 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:47:01.490210 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ea653-a462-4254-a831-247136b85996 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:47:01.548443 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3cb255-cdd6-4623-9fd5-243503d134b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:47:01.549289 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3cb255-cdd6-4623-9fd5-243503d134b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:47:01.549575 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3cb255-cdd6-4623-9fd5-243503d134b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:47:01.549857 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3cb255-cdd6-4623-9fd5-243503d134b8 admin admin] PATH_INFO: `/v3/users/1bea5ab441114aea884879410a735631` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:47:01.551245 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3cb255-cdd6-4623-9fd5-243503d134b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bea5ab441114aea884879410a735631)` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:47:01.553241 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3cb255-cdd6-4623-9fd5-243503d134b8 admin admin] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:47:01.734954 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ac3cb255-cdd6-4623-9fd5-243503d134b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bea5ab441114aea884879410a735631'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:47:01.735641 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ac3cb255-cdd6-4623-9fd5-243503d134b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bea5ab441114aea884879410a735631'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:47:01.743988 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ac3cb255-cdd6-4623-9fd5-243503d134b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bea5ab441114aea884879410a735631'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:47:01.744689 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ac3cb255-cdd6-4623-9fd5-243503d134b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bea5ab441114aea884879410a735631'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:47:01.835864 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.notifications [None req-ac3cb255-cdd6-4623-9fd5-243503d134b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bea5ab441114aea884879410a735631'} {{(pid=80363) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:47:01.854883 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1448/2891] 10.222.0.47 () {58 vars in 1223 bytes} [Thu Jun 30 10:47:01 2022] DELETE /identity/v3/users/1bea5ab441114aea884879410a735631 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:47:02.218078 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb9bb1-7066-4cab-a05a-0c6de942248f admin admin] Authenticating user token {{(pid=80364) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:47:02.218078 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.fernet_utils [None req-9bbb9bb1-7066-4cab-a05a-0c6de942248f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80364) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:47:02.228736 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb9bb1-7066-4cab-a05a-0c6de942248f admin admin] Validating token access rules against request {{(pid=80364) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:47:02.291158 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7a7490-82d1-41d3-bd48-6d000a0b8e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '67e761d82cde42509f040f82a44c86c4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '429c0e046ffb41cbb44280918656fef0', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80364) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:47:02.292160 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a7490-82d1-41d3-bd48-6d000a0b8e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:47:02.292448 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a7490-82d1-41d3-bd48-6d000a0b8e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:47:02.292728 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7a7490-82d1-41d3-bd48-6d000a0b8e44 admin admin] PATH_INFO: `/v3/projects/2fa2b32a2a5145f786d18ab7218a9cf6` {{(pid=80364) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:47:02.293912 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a7490-82d1-41d3-bd48-6d000a0b8e44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fa2b32a2a5145f786d18ab7218a9cf6)` {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:47:02.295859 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7a7490-82d1-41d3-bd48-6d000a0b8e44 admin admin] RBAC: Authorization granted {{(pid=80364) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:47:02.415738 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9e7a7490-82d1-41d3-bd48-6d000a0b8e44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fa2b32a2a5145f786d18ab7218a9cf6', 'request_id': 'req-9e7a7490-82d1-41d3-bd48-6d000a0b8e44'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:47:02.422031 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9e7a7490-82d1-41d3-bd48-6d000a0b8e44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fa2b32a2a5145f786d18ab7218a9cf6', 'request_id': 'req-9e7a7490-82d1-41d3-bd48-6d000a0b8e44'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:47:02.428084 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9e7a7490-82d1-41d3-bd48-6d000a0b8e44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fa2b32a2a5145f786d18ab7218a9cf6', 'request_id': 'req-9e7a7490-82d1-41d3-bd48-6d000a0b8e44'} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:47:02.545311 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9e7a7490-82d1-41d3-bd48-6d000a0b8e44 admin admin] The token cache is being invalidate because project 2fa2b32a2a5145f786d18ab7218a9cf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80364) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 30 10:47:02.545601 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9e7a7490-82d1-41d3-bd48-6d000a0b8e44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:47:02.546255 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG keystone.notifications [None req-9e7a7490-82d1-41d3-bd48-6d000a0b8e44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80364) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 30 10:47:02.547813 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: [pid: 80364|app: 0|req: 1444/2892] 10.222.0.47 () {58 vars in 1232 bytes} [Thu Jun 30 10:47:02 2022] DELETE /identity/v3/projects/2fa2b32a2a5145f786d18ab7218a9cf6 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 30 10:47:11.851154 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=61,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:47:12.541990 dv-cinder-820994-3-iscsi devstack@keystone.service[80364]: DEBUG dbcounter [-] [80364] Writing DB stats keystone:SELECT=51,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80364) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 30 10:47:36.982440 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3cb255-cdd6-4623-9fd5-243503d134b8 admin admin] Authenticating user token {{(pid=80363) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 30 10:47:36.983862 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-ac3cb255-cdd6-4623-9fd5-243503d134b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:47:37.055729 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3cb255-cdd6-4623-9fd5-243503d134b8 admin admin] Validating token access rules against request {{(pid=80363) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 30 10:47:37.116081 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5f0211-f998-4d72-a615-ad6f5a147f86 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46c99f43a9344d7d92632990e45bb9cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '50902e0539434c82b505dc5009aeaf85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80363) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 30 10:47:37.116748 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f0211-f998-4d72-a615-ad6f5a147f86 service placement] REQUEST_METHOD: `GET` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 30 10:47:37.117030 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f0211-f998-4d72-a615-ad6f5a147f86 service placement] SCRIPT_NAME: `/identity` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 30 10:47:37.117306 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5f0211-f998-4d72-a615-ad6f5a147f86 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80363) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 30 10:47:37.119077 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.fernet_utils [None req-3d5f0211-f998-4d72-a615-ad6f5a147f86 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80363) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 30 10:47:37.130468 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f0211-f998-4d72-a615-ad6f5a147f86 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 30 10:47:37.132107 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5f0211-f998-4d72-a615-ad6f5a147f86 service placement] RBAC: Authorization granted {{(pid=80363) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 30 10:47:37.142584 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: [pid: 80363|app: 0|req: 1449/2893] 10.222.0.47 () {60 vars in 1365 bytes} [Thu Jun 30 10:47:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 30 10:47:47.138270 dv-cinder-820994-3-iscsi devstack@keystone.service[80363]: DEBUG dbcounter [-] [80363] Writing DB stats keystone:SELECT=58 {{(pid=80363) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}